by Erica L. Meltzer | Dec 29, 2015 | Blog, The New SAT
Among the partial truths disseminated by the College Board, the phrase “guessing penalty” ranks way up there on the list of things that irk me most. In fact, I’d say it’s probably #2, after the whole “obscure vocabulary” thing.
Actually, calling it a partial truth is generous. It’s actually more of a distortion, an obfuscation, a misnomer, or, to use a “relevant” word, a lie.
Let’s deconstruct it a bit, shall we?
It is of course true that the current SAT subtracts an additional ¼ point for each incorrect answer. While this state of affairs is a perennial irritant to test-takers, not to mention a contributing factor to the test’s reputation for “trickiness,” it nevertheless serves a very important purpose – namely, it functions as a corrective to prevent students from earning too many points from lucky guessing and thus from achieving scores that seriously misrepresent what they actually know. (more…)
by Erica L. Meltzer | Dec 27, 2015 | Blog, The New SAT
In a Washington Post article describing the College Board’s attempt to capture market share back from the ACT, Nick Anderson writes:
Wider access to markets where the SAT now has a minimal presence would heighten the impact of the revisions to the test that aim to make it more accessible. The new version [of the SAT], debuting on March 5, will eliminate penalties for guessing, make its essay component optional and jettison much of the fancy vocabulary, known as “SAT words,” that led generations of students to prepare for test day with piles of flash cards.
Nick Anderson might be surprised to discover that “jettison” is precisely the sort of “fancy” word that the SAT tests.
But then again, that would require him to do research, and no education journalist would bother to do any of that when it comes to the SAT. Because, like, everyone just knows that the SAT only tests words that no one actually uses. (more…)
by Erica L. Meltzer | Dec 26, 2015 | Blog, SAT Essay, The New SAT
For the last part in this series, I want to consider the College Board’s claim that the redesigned SAT essay is representative of the type of assignments students will do in college.
Let’s start by considering the sorts of passages that students are asked to analyze.
As I previously discussed, the redesigned SAT essay is based on the rhetorical essay from the AP English Language and Composition (AP Comp) exam. While they comprise a wide range of themes, styles, and periods, the passages chosen for that test are usually selected because they are exceptionally interesting from a rhetorical standpoint. Even if the works they are excerpted from would most likely be studied in their social/historical context in an actual college class, it makes sense to study them from a strictly rhetorical angle as well. Different types of reading can be appropriate for different situation, and this type of reading in this particular context is well justified.
In contrast, the texts chosen for analysis on the new SAT essay are essentially the type of humanities and social science passages that routinely appear on the current SAT – serious, moderately challenging contemporary pieces intended for an educated general adult audience. To be sure, this type of writing is not completely straightforward: ideas and points of views are often presented in a manner that is subtler than what most high school readers are accustomed to, and authors are likely to make use of the “they say/I say” model, dialoguing with and responding to other people’s ideas. Most students will in fact do a substantial amount of this type of reading in college.
By most academic standards, however, these types of passages would not be considered rhetorical models. It is possible to analyze them rhetorically – it is possible to analyze pretty much anything rhetorically – but a more relevant question is why anyone would want to analyze them rhetorically. Simply put, there usually isn’t all that much to say. As a result, it’s entirely unsurprising that students will resort to flowery, overblown descriptions that are at odds with actual moderate tone and content of the passages. In fact, that will often be the only way that students can produce an essay that is sufficiently lengthy to receive a top score.
There are, however, a couple of even more serious issues.
First, although the SAT essay technically involves an analysis, it is primarily a descriptive essay in the sense that students are not expected to engage with either the ideas in the text or offer up any ideas of their own. With exceedingly few exceptions, however, the writing that students are asked to do in college with be thesis-driven in the traditional sense – that is, students will be required to formulate their own original arguments, which they then support with various pieces of specific evidence (facts, statistics, anecdotes, etc.) Although they may be expected to take other people’s ideas into account and “dialogue” with them, they will generally be asked to do so as a launching pad for their own ideas. They may on occasion find it necessary to discuss how a particular author presents his or her evidence in order to consider a particular nuance or implication, but almost never will they spend an entire assignment focusing exclusively on the manner in which someone else presents an argument. So although the skills tested on the SAT essay may in some cases be a useful component of college work, the essay itself has virtually nothing to do with the type of assignments students will actually be expected to complete in college.
By the way, for anyone who wants understand the sort of work that students will genuinely be expected to do in college, I cannot recommend Gerald Graff and Cathy Birkenstein’s They Say/I Say strongly enough. This is a book written by actual freshman composition instructors with decades of experience. Suffice it to say that it doesn’t have much to do with what the test-writers at the College Board imagine that college assignments look like.
Now for the second point: the “evidence” problem.
As I’ve mentioned before, the SAT essay prompt does not explicitly ask students to provide a rhetorical analysis; rather, it asks them to consider how the writer uses “evidence” to build his or her argument. That sounds like a reasonable task on the surface, but it falls apart pretty quickly once you start to consider its implications.
When students do the type of reading that the SAT essay tests in college, it will pretty much always be in the context of a particular subject (sociology, anthropology, economics, etc.). By definition, non-fiction is both dependent on and engaged with the world outside the text. There is no way to analyze that type of writing meaningfully or effectively without taking that context into account. Any linguistic or rhetorical analysis would always be informed by a host of other, external factors that pretty much any professor would expect a student to discuss. There is a reason that “close reading” is normally associated with fiction and poetry, whose meanings are far less dependent on outside factors. Any assignment that asks students to analyze a non-fiction author’s use of evidence without considering the surrounding context is therefore seriously misrepresenting what it means to use evidence in the real world.
In college and in the working world, the primary focus is never just on how evidence is presented, but rather how valid that evidence is. You cannot simply present any old facts that happen to be consistent with the claim you are making – those facts must actually be true, and any competent analysis must take that factor into account. The fact that professors and employers complain that students/employees have difficulty using evidence does not mean that the problem can be solved just by turning “evidence” into a formal skill. Rather, I would argue that the difficulties students and employees have in using evidence effectively is actually a symptom of a deeper problem, namely a lack of knowledge and perhaps a lack of exposure to (or an unwillingness to consider) a variety of perspectives.
If you are writing a Sociology paper, for example, you cannot simply state that the author of a particular study used statistics to support her conclusion, or worse, claim that an author’s position is “convincing” or “effective,” or that it constitutes a “rich analysis” because the author uses lots of statistics as evidence. Rather, you are responsible for evaluating the conditions under which those statistics were gathered; for understanding the characteristics of the groups used to obtain those statistics; and for determining what factors may not have been taken into account in the gathering of those statistics. You are also expected to draw on socio-cultural, demographic, and economic information about the population being studied, about previous studies in which that population was involved, and about the conclusions drawn from those studies.
I could go on like this for a while, but I think you probably get the picture.
As I discussed in my last post, some of the sample essays posted by the College Board show a default position commonly adopted by many students who aren’t fully sure how to navigate the type of analysis the new SAT essay requires – something I called “praising the author.” Because the SAT is such an important test, they assume that any author whose work appears on it must be a pretty big. As a result, they figure that they can score some easy points by cranking up the flattery. Thus, authors are described as “brilliant” and “passionate” and “renowned,” even if they are none of those things.
As a result, the entire point of the assignment is lost. Ideally, the goal of close reading is to understand how an author’s argument works as precisely as possible in order to formulate a cogent and well-reasoned response. The goal is to comprehend, not to judge or praise. Otherwise, the writer risks setting up straw men and arguing in relation to positions that they author does not actually take.
The sample essay scoring, however, implies something different and potentially quite problematic. When students are rewarded for offering up unfounded praise and judgments, they can easily acquire the illusion that they are genuinely qualified to evaluate professional writers and scholars, even if their own composition skills are at best middling and they lack any substantial knowledge about a subject. As a result, they can end up confused about what academic writing entails, and about what is and is not appropriate/conventional (which again brings us back to They Say/I Say).
These are not theoretical concerns for me; I have actually tutored college students who used these techniques in their writing.
My guess is that a fair number of colleges will recognize just how problematic an assignment the new essay is and deem it optional. But that in turn creates an even larger problem. Colleges cannot very well go essay-optional on the SAT and not the ACT. So what will happen, I suspect, is that many colleges that currently require the ACT with Writing will drop that requirement as well – and that means highly selective colleges will be considering applications without a single example of a student’s authentic, unedited writing. Bill Fitzsimmons at Harvard came out so early and so strongly in favor of the SAT redesign that it would likely be too much of an embarrassment to renege later, and Princeton, Yale, and Stanford will presumably continue to go along with whatever Harvard does. Aside from those four schools, however, all bets are probably off.
If that shift does in fact occur, then no longer will schools be able to flag applicants whose standardized-test essays are strikingly different from their personal essays. There will be even less of a way to tell what is the result of a stubborn 17 year-old locking herself in her room and refusing to show her essays to anyone, and what is the work of a parent or an English teacher… or a $500/hr. consultant.
by Erica L. Meltzer | Dec 18, 2015 | Blog, SAT Essay, The New SAT
In response to my previous post on the equity issues surrounding the redesigned SAT essay, one reader had this to say:
I read a few of the essay prompts and honestly they seem like a joke. Essentially, each prompt asked for the exact same thing, it’s almost like CB is just screaming “MAKE A TEMPLATE” because all students need to do is plug in the author’s name, cite an example, and put a quote here and there and if it’s surrounded by prepared fancy sentences, they’ve got an easy 12. (or whatever it is now)
That’s a fair point. I didn’t actually mean to imply in my earlier post that students would actually need to be experts in rhetoric in order to score well – my goal was primarily to point out the mismatch between the background a student would need to seriously be able to complete the assignment, and the sort of background the most students will actually bring to the assignment.
For a good gauge of what is likely to happen, consider the French AP exam, which was revised a couple of years ago to be more holistic and “relevant.” It now includes a synthesis essay that is well beyond what most AP French students can write. The result? Score inflation. A similar phenomenon is inevitable here: when there is such a big mismatch between ideal and reality, the only way for the College Boart to avoid embarrassment and promote the illusion that students are actually doing college-level work is assign high scores to reasonably competent work that does not actually demonstrate mastery but that throws in a few fancy flourishes, and solid passing scores to work that is only semi-component.
So I agree halfway. Something like what the reader describes is probably going to be a pretty reliable formula, albeit one that many students will need tutoring to figure out. But that said, I suspect that it will be one for churning out solid, mid-range essays, not top-scoring ones. Here’s why:
While looking through the examples provided by the College Board, I noticed something interesting: out of all the essays, exactly one made extensive use of “fancy” rhetorical terminology (anecdote, allusion, pathos, dichotomy). Would you like to guess which one? If you said the only essay to earn top scores in each of the three rubric categories, you’d be right.
What this suggests to me is that the redesigned essay will in fact be vulnerable to many of the same “inflation” techniques that many high-scoring students already employ. As Katherine Beals and Barry Garelick’s recent Atlantic article discussed, the only way to assess learning is to look for “markers” typically associated with comprehension/mastery. A problem arises, however, when the goal becomes solely to exhibit the markers of mastery without actually mastering anything – and standardized test- essays are nothing if not famous for being judged on markers of mastery rather than on substance.
The current SAT essay, of course, has been criticized for encouraging fake “fancy” writing – bombastic, flowery prose stuffed full of ten-dollar words, and there is absolutely nothing to suggest that this will change. In fact, the new essay is likely to encourage that type of writing just as much, if not more, than the old one.
Indeed, the top-scoring examples include some truly cringe-worthy turns of phrase. For example, consider one student’s statement that “This dual utilization of claims from two separate sources conveys to Gioia’s audience the sense that the skills built through immersion in the arts are vital to succeeding in the modern workplace which aids in logically leading his audience to the conclusion that a loss of experience with the arts may foreshadow troubling results.”
Not to mention this: “In paragraph 5, Gioia utilizes a synergistic reference to two separate sources of information that serves to provide a stronger compilation of support for his main topic” (https://collegereadiness.collegeboard.org/sample-questions/essay/2, last example).
And this: “In order to achieve proper credibility and stir emotion, undeniable facts must reside in passage.”
This is the sort of prose that makes freshman (college) composition instructors tear their hair out. Is this what the College Board means by “college readiness?”
Furthermore, if the use of fancy terminology correlates with high scores, why not exploit that correspondence and simply pump out essays stuffed to the gills with exotic terms, with little regard for whether they describe what is actually occurring in the text? As long as the description is sufficiently flowery, those sorts of details are likely to slip by unnoticed.
In fact, why not go a step further and simply make up some Greek-sounding literary terms? Essay graders are unlikely to spend more than the current two minutes scoring essays; they don’t have the time or the liberty to check whether obscure rhetorical terms actually exist. Some really smart kids with a slightly twisted sense of humor will undoubtedly decide to have some fun at the College Board’s expense. Heck, if I could force myself to wake up at 6 a.m. on a Saturday morning, I’d be tempted to go and do it myself.
Another observation: as discussed in an earlier post, one of the key goals of the SAT essay redesign was to stop students from making up information. But even that seems to have failed: lacking sufficient background information about the work they are analyzing, students will simply resort to conjecture. For example, the writer of the essay scoring 4/3/4 states that Bogard (the author of the first sample passage) is “respected,” and that “he has done his research.” Exactly how would the student know that Bogard is “respected?” Or that he even did research? (Maybe he just found the figures in a magazine somewhere.) Or that the figures he cites are even accurate?
It is of course reasonable to assume those things are true, but strictly speaking, the student is “stepping outside the four corners of the text” and making inferences that he or she cannot “prove” objectively. So despite the College Board’s adamant insistence that essays rely strictly on the information provided in the passages, the inclusion of these sorts of statements in a high-scoring sample essay certainly suggests that the boundary between “inside” and “outside” the text is somewhat more flexible than it would appear.
The point, of course, is that it is extraordinarily difficult, if not downright impossible, to remain 100% within the “four corners” of a non-fiction text and still write an analysis that makes any sense at all – particularly if one lacks the ability to identify a wide range of rhetorical figures. Of course students will resort to making up plausible-sounding information to pad their arguments. And based on the sample essays, it certainly seems that they will continue to be rewarded for doing so.
by Erica L. Meltzer | Dec 13, 2015 | Blog, SAT Essay, The New SAT
In my previous post, I introduced some of background issues surrounding the new SAT essay. Here, I’d like to examine to examine how the redesigned essay, rather than make the SAT a fairer test as the College Board claims, will likely provide further advantages to a small, already privileged segment of the test-taking population.
Let’s start with the fact that the new essay was adapted from the rhetorical essay on the AP English Language and Composition (AP Comp) exam.
To be sure, the colleges that require the essay are likely to draw many applicants enrolled in AP Comp as juniors, but there will still undoubtedly be tens of thousands of students sitting for the SAT essay who are not enrolled in AP Comp, or whose schools do not even offer that class. Many other students whose schools do offer AP Comp may not take it until senior year, well after they’ve taken their first SAT. And then there will be students who are enrolled in AP Comp as juniors but who spend class talking in small groups with their classmates and not ever being taught about rhetorical analysis. (Based on my own experience tutoring AP Comp, I suspect that many students fall into that last category.) What sort of value is there in asking so many students to write an essay that they are totally unprepared to write?
The College Board, of course, has attempted to make the essay appear more egalitarian by insisting that no particular terminology or background knowledge is required, and that students can find all the information they need in the passage. Whereas the AP Comp prompt explicitly asks students to analyze the rhetorical choices…an author makes in order to develop her/her argument, the College Board deliberately avoids use of the term “rhetoric” for the SAT essay, opting for the more neutral “evidence and reasoning” and stylistic or persuasive elements,” and giving the impression that the assignment is wide open by asserting that students may also analyze features of [their] own choice.
The problem is that this type of formal, written textual analysis is a highly artificial task. It involves a very particular type of abstract thought, one that really only exists in school. (I suspect that the only American students seriously studying rhetoric at a high level are budding classicists at a handful of very, very elite mostly private schools – a minuscule percentage of test-takers.) Learning to notice, to divide, to label, to categorize, to enter into a text and describe the order and logic by which it functions… These are not instinctive ways of reading; learning to do these things well takes considerable practice. If students don’t acquire this particular skill set in school, and more particularly in English class, they almost certainly won’t acquire it anywhere else. Of all the things tested on the new SAT, this is the one Khan Academy is least equipped to handle.
It also strikes me as naïve and more than a little bit misleading to insist that this type of analysis can be performed by any old student if the technical aspect is removed – that is, if students write “appeal to emotion” rather than pathos. Yes, it is possible to write a top-scoring essay that employs nothing but plain old Anglo-Saxon words, but in most cases, the students most capable of even faking their way through a rhetorical analysis will be precisely the ones who have learned the formal terms. Students don’t somehow acquire skills simply from being allowed to express themselves in non-technical language.
The choice to adopt this particular template for the SAT essay was, I imagine, based on the presumption that Common Core would sweep through classrooms across the United States, with all students spending their English periods diligently practicing for college and career readiness by combing through non-fiction passages, finding “evidence” (the text means what it means because it says what it says) and identifying “appeals to emotion and authority.”
Needless to say, that reality has not materialized; however, the SAT is based on the assumption that it would.
So whereas smart, solid writers who didn’t do much of anything in English class had as good a chance as anyone else at doing well on the pre-2016 essay, smart, solid writers who have not been given practice in this particular type of writing in English class will now be at a much larger disadvantage. There will of course be the extreme outliers who can sit themselves down with a prep book, read over a few examples, and start churning out flawless expository prose, but they will be the exceptions that prove the rule.
As for the rest, well… let’s just say the new essay is basically the College Board’s gift to the tutoring industry.
