As I was categorizing the reading questions from the new tests in the 2020 edition of the Official SAT Guide, I noticed something a little odd about question #47 from the October 2017 exam.

The question, which accompanies a passage about the search for new types of antibiotics, reads as follows:

 

In line 79, “caveats” most nearly means

A) exceptions
B) restrictions
C) misgivings
D) explanations

 

Now, the answer, C), was correct in the most technical sense. Among the answer choices, “misgivings” obviously made the most logical sense when it was plugged into the passage, and it was perfectly consistent with the list of drawbacks the author provided in regard to a particular drug.

But when I thought about it, something about the question kept nagging at me.

“Caveats” are warnings, full stop; the word only has one definition. “Misgivings,” in contrast, are feelings of doubt, or reservations.

The two are related in the sense that an author would logically only offer caveats if he had misgivings about something; however, to say that the two are the same, or that one can be used in place of the other without significantly changing the connotation, is simply not true. In fact, it’s exactly the sort of questionable justification for a wrong answer my students used to offer for vocabulary questions on the old SAT: “Well, x can sort of mean y, because y happens because of x…” Or, “They’re sort of similar, so they’re kind of like saying the same thing.” (And to push the point even further, the ability to distinguish between x causes y and x is y is precisely what SAT analogies used to test. The ability to make strong arguments is in part dependent on the ability to make these kinds of distinction.)

Furthermore, there is nothing in the passage to firmly indicate that “caveats” is being used to mean anything other than what it normally means. In fact, there is no information in the passage that makes clear the precise meaning of “caveats” at all. True, “warnings”—and, to be honest, “caveats”—does not sound nearly as good as “misgivings” in context; the original usage is somewhat awkward. However, “sounds good” should not be confused with “has the same connotation.”

Likewise, the fact that a word happens to make sense in context does not imply that it is what the author actually means. (Apparently, that was too subtle a point for the test-writers; based on the College Board’s explanation, this is where they seem to have gotten tripped up.) To understand what the writer is suggesting—namely, that others should beware of viewing teixobactin as a cure-all, not just that he personally has reservations about its use—the reader simply has to know the literal definition of “caveats.”

The use of the first-person (my) in the sentence in which the word appears, as well as in the following sentence (“Well, I see three [caveats]), does not affect that connotation; it is in no way inconsistent with the word’s usual meaning to emphasize that one is personally offering a warning. This straightforward interpretation of the word is also supported by the description of substantial logistical hurdles that follows: the whole point is that other researchers would be mistaken to place too much faith in teixobactin.

Besides, writers don’t use fancy words like “caveats” by accident. If the author of the passage had wanted to indicate that he was merely expressing a personal opinion, he wouldn’t have gone out of his way to use a term with an explicit connotation to the contrary.

I’ve encountered problematic questions before, and beyond some fleeting irritation, I don’t normally waste much time thinking about them. For some reason, though, the wrongness of this question kept gnawing at me, and I couldn’t seem to stop thinking about it.

The question was sloppy, sure, but the problem seemed to go beyond just a lack of precision.

It took me a while to put my finger on what was bothering me so much, but the more I turned things over in my mind, the more problems I noticed, and the deeper they went.

What I finally realized was that in addition to encouraging a misreading of the text, the question seemed to fundamentally miss the point of what vocabulary-in-context questions are intended to test, in a way I’d never really seen on the old SAT, the ACT, the TOEFL, or any graduate-level exam.

Worse, it was based on a refusal to recognize that definitions cannot always be determined from context, and that because English is a hybrid language, different types of words often require different types of approaches.

In short, the question was messed up in practice because it was messed up in theory.

Let me start here: Normally (that is, on pretty much every competently written standardized test), vocabulary-in-context questions exclusively concern alternate meanings of common words, and are intended to test whether students have grasped the actual meaning of a text.

Although they ask what a term “most nearly means,” that phrasing is not intended to imply that the correct answer is an approximation of the definition, nor is it an invitation for the test-taker to change the meaning of the text. Rather, it is a signal that the word being tested is not being used in its most common definition. The correct answer is the option that corresponds most precisely to the author’s intended meaning or connotation.

Because words with multiple meanings by definition have, well, more than one meaning, it is unnecessary for the test-writers to grope around for a synonym that sort of kind of fits as the answer choice; the alternate definition, which is often quite common and conveys the actual meaning, can simply be plugged in.

For example, consider the following question:

 

Over the last several years, farmers in Florida have attempted to check the spread of the crop disease known as citrus greening, responsible for destroying millions of tons of fruit, by spraying orange, grapefruit, and lemon trees with a variety of pesticides.

In line 1, “check” most nearly means”

A) reveal
B) limit
C) defend
D) comprehend

 

The answer here is B) because “limit” is not a vague approximation of the meaning—it is exactly what “check” means here. That usage just happens to be less common than the usual definition, i.e., “make sure.”

A question like this will automatically be challenging for some students because they will be either unfamiliar with that alternate meaning and/or unable to use context clues to figure it out. It is unnecessary to complicate things further.

When a term only has one literal definition, however, this game is impossible to play: first because the whole point of these questions is to test words with multiple meanings, and second because there may be no second option to use as the correct answer.

“Caveat” is a perfect illustration of a term in this category: it has exactly one meaning— “warning”—that has no exact synonyms. But because the literal definition cannot appear as the correct answer, lest the College Board give the impression that the SAT is a test of simple memorization, a less precise alternative must be used. Hence “misgivings.”

But rather than serve to reveal whether a student has understood the text’s literal meaning, as these questions are intended to do, the answer changes the text’s meaning. So while the question is acceptable in the absolute, literal sense it also violates the spirit of what vocabulary-in-context questions are all about. Basically, the test-writers exploited a loophole to produce a misleading question that still technically passed muster.

It is entirely possible to imagine a student who knows the literal definition of “caveats”—and who thus understands the text perfectly—getting stuck here because there is no synonym for “warnings” among the choices. And again, that is a problem because aside from the slightly awkward phrasing, nothing in the text directly signals that “caveats” is being used to mean anything other than what it usually means. As a result, it is necessary to focus not on the word itself, but rather on the answer choices. The test-taker must read between the lines and intuit what the test-writers were attempting to get at, as opposed to what they literally asked. Plenty of smart kids don’t get how to play that game.

Alternately, a savvy student could ignore “caveats” altogether and focus on the answer that produces the most logical meaning in context—that’s pretty much the simplest way to answer the question. (One can easily imagine a well-coached student screwing up his/her nose at the appearance of a “weird” word, then remembering a tutor’s advice to forget about the original term and just focus on the answer choices.) But even if that method leads to a correct answer, the student will have misconstrued the author’s actual intent.

This is basically the high school equivalent of telling children who are learning to read that it’s okay to see the word “horse” and instead read “pony” because “pony and horse mean the same thing.”

The message is: it doesn’t really matter whether you know what words actually mean, as long as you’re sort of somewhere in the ballpark.

The consequences for a single question like this are admittedly trivial, but as a general approach to vocabulary acquisition, it’s well, kind of a problem.

At any rate, the fact that the question did appear on an administered test essentially leaves two possibilities:

1) The test-writers, as well the people higher up responsible for reviewing their work, did not really grasp what these questions were intended to test, and/or mistook the awkward phrasing in the passage for an indication that “caveats” was not being used literally.

This is pretty probable, given the College Board’s decision to push out ETS and take over the test-writing process itself, along with its replacement of long-time employees with weaker, less experienced new hires, largely from the ACT (a test that does not target advanced vocabulary). In fact, by the end of 2015, three quarters of the top leadership had worked at College Board for less than two years. It is not unreasonable to assume that the percentage is even higher now.

Undoubtedly, there are many words just as challenging as “caveats” that have not one but two close synonyms, and with a bit more effort, the test-writers could have found a better candidate for a question. (To choose a random example, “perforated,” in line 29 of the same passage for example, can be used to mean either “punctured” or “pierced.”) The fact that they did not do so is very telling.

2) No one really cared. (Because, hey, as long as we throw in some stuff that looks kinda hard, it’s all good.)

Note that these two options are not mutually exclusive.

I think, however, that the real problem is the framework underlying the creation of the question.

Recall that a significant part of the marketing campaign surrounding the redesigned SAT revolved around the fact that words would only be tested in context—that is, “authentically”. No more would students be made to lug around stacks of flashcards bearing lists of polysyllabic monstrosities never to be encountered in real life (like, for example, “caveats”). No, the redesigned SAT would only test “relevant” words—an Orwellian formulation if ever there was one.

As the ACT has discovered recently, however, if you don’t include enough difficult questions, or fail to manipulate the curve properly, too many people get perfect scores. In contrast, the redesigned SAT was designed so that scores in the middle range would increase; the number of perfect scores needed to be capped in order to avoid the perception that the test had been made too easy, and the test still needed to give the impression of rigor.

This, then, is the College Board trying to have it both ways: the test needs to include some advanced vocabulary so that it maintains its status among its traditional clientele of high-achieving students and their parents/college counselors, but the organization has also backed itself into a corner with the “words in context” paradigm.

Thanks to the College Board’s own marketing campaign, the test must now adhere to the idea that even “hard” words can be figured out if the reader has mastered the correct strategy, i.e., using context clues. Unfortunately, that is not how reality works. And on a question like the one I’ve been discussing, the limitations of this approach become apparent.

In fact, unless you are repeatedly exposed to a relatively uncommon word like “caveat” in situations that make its meaning clear (highly unlikely for the average high schooler), there are exactly two ways to learn it:

1) You can memorize it purely by rote.

2) You can use the Latin root “cave” (“beware,” pronounced CAH-way), which makes it considerably easier to remember, especially if you happen to be studying Latin.

Obviously, the student who learns it via option #2 stands an infinitely better chance of retaining it, new SAT or old SAT, vocabulary flashcards or not. But the College Board’s entire spiel is based on a denial of the fact that this is a legitimate way to learn vocabulary—or, at the very least, a way that should actively be encouraged. Presumably, this has something to do with the fact that the opportunity to study Latin—or, heaven forbid, Greek—is largely only available to more privileged students, which puts it at odds with the College Board’s mission of “expanding opportunity.”

But if the goal is to test some higher-level vocabulary effectively (admittedly a shaky premise), then this is a serious problem. Because of the way the English language evolved, many—if not most—single-meaning challenging/technical terms, are indeed derived from Latin or Greek, in contrast to everyday words, which tend to have Anglo-Saxon origins. In addition, higher-level Greco-Latin terms often appear in circumstances in which their exact meanings cannot be determined from context.

As a result, students who do not know anything about roots or etymologies lack an important set of tools for learning and retaining new words, particularly the kinds of words they are likely to encounter in college. (Yes, this is even true for many students in STEM fields, who are likely to encounter countless scientific terms derived from Latin!)

That is why the old SAT tested vocabulary from both angles, and in two different types of exercises: sentence completions were designed so that many answers could be determined via roots, whereas passage-based questions depended on context clues. To some extent, that division reflected the fact that English is a hybrid language, and that words can (and, in some cases, must) be learned in a variety of ways. But if the College Board was going to regain market share from the ACT, that reality needed to be rejected wholeheartedly.

To return to my original point, though, the last piece of why the “caveats” question bothers me so much has to do with the way in which “misgivings” shifts the connotation of the text from the social to the personal. I do not want to imply that this was anything other than a random coincidence on the College Board’s part, but I also find it oddly telling that the answer involved this particular type of alteration.

Let me explain.

Increasingly, I am encountering people—not just teenagers, but also some adults—who seem not just unwilling but actually unable to conceive of ideas in general terms, instead reducing everything to personal opinion and/or experience and refusing to acknowledge any external view. I think it’s reasonable to assert that this tendency is reinforced, perhaps even developed, by a school system in which students are constantly encouraged to relate what they are studying to their own lives.

This is obviously not a bad idea in moderation, but like anything, it becomes a real problem when applied simplistically and to extremes. The result is that everything is reduced to a matter of individual opinion and experience, and any criticism of a person’s ideas is perceived as a personal attack on their unique identity (reductio ad hominem?).

In that context, the replacement of “caveats” with “misgivings” slips the author’s discussion onto the plane of personal opinion—“Oh, well, he personally thinks that this stuff, tex-, tex-whatever, won’t work, but that’s just, like, his individual perspective,” one can imagine a particular type of student stating smugly—and away from an act directed toward others.

I understand if this reading of the question seems like too much of a stretch to some people, but given some of the incidents I’ve witnessed recently—not least, a middle-schooler haranguing a seasoned researcher of reading disabilities in front of a room full of adults, obstinately insisting that he could not possibly understand what it meant to be dyslexic since he himself was not personally gifted (!) with dyslexia—I can’t help but think about it in these terms. In that context, it seems to epitomize an enormous issue at a very minute scale.

For me, the bottom line is this: some words do have generally agreed-upon definitions, and to pretend otherwise is to invite a world in which coherent discussion is impossible because what should be common ground can always shift—one of the parties can always insist that terms mean other than what they mean; that because they really believe it, their interpretation must be taken seriously; and that to insist otherwise is a threat to their very selves.

Indeed, if everyone is an entirely unique individual entitled to interpret everything in their own way, there can be no such thing as a “commonsense” understanding because there can be no such thing as “common.”

Thus, sense becomes nonsense, and nonsense sense.

I think it’s fair to say that’s a real problem.

But then again, maybe those are just my personal misgivings.