Over the past few weeks, the test-optional dominos have continued to fall, with Harvard grudgingly deciding to consider applications from students who have faced exceptional obstacles in taking the SAT or ACT, and Princeton even more grudgingly following suit. As of now, the Ivies seem pretty clear about the fact that these are one-year policies only, and that applicants applying in the fall of 2021 and beyond will be expected to take the tests as usual.

At other other selective colleges, however, this year’s policies are part of a multi-year test-optional trial period, and so I think it’s worth taking a hard look at the implications of these policies in a non-Covid context, and to ask who really benefits from them.

As readers of this article may well know, the University of Chicago made waves in 2018 when it became the first elite research university to implement a test-optional policy as part of its Empower program. (Among the program’s other measures were increased recruitment and financial aid for first-generation students, rural students, children of police and firefighters, and veterans—if nothing else, Chicago’s somewhat idiosyncratic definition of “underrepresented” exemplifies its savvy at warding off accusations of pandering to political correctness.) A year later, in July of 2019, the university released a statement proclaiming the initiative a success: 

The number of first-generation and low-income students who committed to attend UChicago has increased 20 percent, and enrollment of rural students has grown 56 percent. Empower scholarships and access programs for those who serve helped produce growth in the enrollment of veterans and the children of police officers and firefighters from across the nation. The expanded outreach broadened the impact of UChicago’s long-established financial aid and programmatic resources for students of high ability from underrepresented communities, resulting in a continued increase this year in the enrollment of African American and Hispanic/Latino students.

 

An expansion of UChicago’s commitment to access and inclusion, a core principle since the University’s founding in 1890, Empower was created to enable more students to pursue higher education, regardless of background, geographic location or ability to pay. The program increased financial aid and outreach efforts, and made UChicago the first highly selective college to make standardized tests optional in the application process. The significant enrollment growth supports the importance of intentional outreach to talented, ambitious students in underserved communities who might not otherwise aspire to attend a highly selective college.

The clear implication here is that the University was able to admit a more diverse class as a result of the shift to making test scores optional, although—significantly—this is not explicitly stated. Indeed, this is the usual spiel when it comes to the claims that colleges make about the effects of their new test-optional policies. So I’d like to start by putting these numbers in context and examining them a little more closely—call it a a case study in the relationship between rhetoric and reality.

Writing in Inside Higher Ed, Scott Jaschik pointed out in response to Chicago’s announcement, the university could hardly have done anything other than proclaim the success of its program. Having made such bold statements, the university couldn’t exactly have walked back from them.

Likewise, another article in that publication makes the point that many schools that did NOT introduce test-optional policies have seen comparable increases in the number of first-generation/URM applicants. So in the absence of specific evidence indicating that students in these demographics decided to apply to Chicago based on its test-optional policy, and that they actually applied and were then accepted under that policy, it’s impossible to tell whether the jump is a matter of correlation or causation.

Furthermore, when you look at where Chicago stands relative to its peers, it quickly becomes apparent how modest the gains really are.

In terms of its student body’s socioeconomic profile, U. Chicago actually lags well behind the Ivies in the percentage of its students receiving Pell grants (the standard measure of economic disadvantage). In 2017-2018, the year in which the new policy was announced, only 11% of U. Chicago undergraduates were Pell grant recipients, as compared to 15% at Harvard, 17% at MIT and Columbia, and 21% at Princeton (which to its credit has made a massive push to recruit low-income students in recent years).

In terms of first-generation students, U. Chicago, with 14%, again lags behind Yale (22%), Columbia (23%), Princeton (25%), and UCLA/UC Berkeley at 35% and 33% respectively. So far from being a leader in social mobility, Chicago is in fact playing a very substantial game of catchup—something that was effectively omitted from most discussions regarding the announcement of its program. (Washington Monthly ranks it 37th on its 2020 mobility rankings; Pell Grant statistics downloadable from the Washington Monthly website.)

Given that the U. Chicago class of 2022 includes 1806 students, a 20% increase (for the class of ’23) over that 11/14% (about 225 students) is actually not a particularly high number—somewhere around 45 students.

Furthermore, the press release cannily elides “low-income” and “first-generation,” without providing the exact breakdown. While there is certainly a correlation between the two groups—salaries for non-college graduates being generally lower than those for college graduates—affluent first-generation students certainly do exist. As a result, more information is required to understand how many of those 45 or so students are genuinely disadvantaged.

The same is true for rural students, whose presence in the class of 2023 increased by 56%. That sounds like a large number, but the problem is that the press release provides no basis for comparison. Is that an increase of 25 students? 50? 100?

Moreover, the same issue is at play for rural students as for first-generation students: living in a relatively unpopulated area and coming from an affluent background are by no means mutually exclusive (nor is living in a rural area mutually exclusive with receiving a high score on the SAT or ACT). Under-represented does not necessarily mean “disadvantaged.” In the general population, there may indeed be a correlation; however, we are talking about a very small, select number of applicants to an elite college—a group that by definition is rife with exceptions.

Presumably U. Chicago did in fact admit some low-income rural applicants whose test scores would have given the admissions committee pause in previous years. But again, without a breakdown of the variables, there’s no way to tell how significant that number actually is. (And interestingly, 2023 SAT scores range from 1060 to 1600, and ACT scores from 20-36, indicating that exceptional applicants with sub-par scores are still being admitted regardless.)

Significantly, the press release refers only vaguely to an “increase” in the number of African American and Hispanic/Latino students, and the mention is buried toward the bottom of the first paragraph. Reading between the lines, it’s fair to assume that the numbers were quite small—had they been more substantial, the university would have gone out of its way to tout them. If anything, they suggest that the typically invoked justification for test-optional falls short.

While I was poking around online trying to find more specific information about this, I came across a U. Chicago student-newspaper article written by a freshman who had been essentially forced to apply test-optional as a result of a horrific series of operations she had to undergo her junior year of high school. Indeed, her story makes an compelling case for flexibility regarding standardized testing in extreme circumstances.

But although the writer herself argues in favor of test-optional by rehashing the standard talking points about equity, it’s hard not to wonder whether she is in fact more typical of the test-optional applicants actually being admitted: a presumably non-disadvantaged, non-first-generation graduate of a a small prep school that sends a large percentage of its students to elite colleges, and who acquired significant university-level scientific research experience while still in high school. (Yes, I went and looked up her LinkedIn profile.)

How many students in the 10-15% percent of U. Chicago’s class of 2023 that applied test-optional look like her vs. a more stereotypically disadvantaged student?

Looking at schoools beyond U. Chicago, it is interesting to note that of the colleges that have moved to test-optional policies within the last few years, Middlebury is the only other one with need-blind admissions. (Hamilton, which is also need-blind, has a “test-flexible” policy.”) On the other hand, Wesleyan made the switch from need-blind to need-aware in 2012 and went test-optional just two years later.

Furthermore, the schools that have practiced need-blind admissions the longest—Bates, Bowdoin, and Colby—are hardly paragons of diversity: at Bates and Colby, less than 5% of the student body is African American and only around 7-8% Hispanic. Bowdoin does slightly better, with about 6.5% African American and 8% Hispanic. In contrast, the numbers at non-test optional Amherst, which is known for actively recruiting underrepresented minority applicants, are almost double Bowdoin’s.

To be sure, there are geographical considerations at play: the small-town Maine locations of Bates, Bowdoin, and Colby have less broad appeal than Amherst, which is located in a popular college town and offers cross-registration with four other institutions. But at this point, the Maine schools have had several decades to improve their numbers; if test-optional policies alone were going to fix the diversity problem, they would have done so already. Instead, the colleges continue to attract disproportionate numbers of prep schoolers and affluent suburbanites who aren’t quite strong enough for the Ivies.

Colby in particular is notable for having a fifth of its student body hail economically from the top 1% ($630K+), whereas only half that number comes from the entire bottom 60%.

At Middlebury, a much newer member of the test-optional club, 22% came from the top 1% versus 14.2 from the bottom 60% in 2017.

So if the true goal were to have an honest discussion about which groups benefit most from test-optional policies, then it would be helpful to know the answers to questions such as the following:

 

• What proportion of students admitted without scores attend public vs. private schools?

 

• Of the non-submitting students admitted from private schools, how many come from traditional (elite) feeders that reliably send multiple students to that institution every year? Are there particular elite high schools whose students are repeatedly admitted under test-optional policies?

 

• What are the average and median family incomes of students admitted without scores?

 

• How does the average family income of first-generation students admitted without scores compare to that of non-first-gen applicants admitted without scores?

 

• What percentage of students admitted without scores are full-pay? How does the admission rate of fully-pay applicants compare to ones who require financial aid?

 

• What percentage of students admitted without scores are legacies?

 

 

Now, it is possible that going test-optional does in fact lead to an increase (however marginal) in the number of URM applicants/admits without giving a concomittant advantage to certain types of highly advantaged traditional applicants.

It is also possible that going test-optional results in an increase in URM applicants/admits while also giving an equal or greater advantage to some highly privileged traditional applicants.

But the point is that we don’t know because no one pushes for the answers to these questions, and in any case, colleges would never release that information (at least not in the absence of a court order).

And what really concerns me even more is that even if they did release it, any information that contradicted the standard line that eliminating required testing benefits URM students would almost certainly be either suppressed or ignored—the latter of which is precisely what happened to the University of California’s faculty-panel findings indicating that test scores actually benefitted many applicants from under-represented groups.

This is really incredibly unhealthy: decisions at a university, of all places, should be informed by research, and officials should be willing to consider evidence that goes against prevailing beliefs (provided, of course, that it’s the result of an objective, well-designed investigation).

Finally, allow me to point out that in absolute terms, there is no reason for test scores to have any effect whatsoever on the composition of a freshman class. Under the holistic admissions process, selective colleges are effectively free to select their classes according to any particular criteria they happen to value—there is no such thing as minimum required scores (as U. Chicago’s score ranges very well demonstrate).

The problem, of course, is the USNWR rankings which, despite some rejiggering to the formula to emphasize social mobility, continue to hold enormous sway over colleges’ decisions surrounding standardized testing. As a long as scores are factored into the rankings formula, schools will be incentivized to inflate their numbers through any means possible. At the same time, the additional weight placed on social mobility (an alteration that caused U. Chicago to drop three spots to #6 in the 2020 rankings) will drive them to take new measures to attract underprivileged students. More than anything else, test-optional policies offer the best of both worlds.