Manuel Alfaro, a former executive director at the College Board, has written a series of posts on LinkedIn detailing the myriad problems plaguing the development of the new exam.
According to Alfaro, not only were many of the items developed for the first administration of the test extraordinarily problematic (see below), but many of the items that appeared on the test were not actually reviewed by the Content Advisory Committee until after the test forms had been constructed.
Committee members repeatedly attempted to call David Coleman’s attention to the problem, but were ignored.
Alfaro was also responsible for rewriting and rubber-stamping the redesigned test specifications in order to hide the fact that they were taken directly from the Common Core middle- and high school standards. (Because, of course, the College Board expected everyone to somehow forget that David Coleman, the head of the College Board, was also responsible for Common Core.)
The College Board completed the backstory for the test specifications by citing reports/analyses performed by independent groups as evidence of the alignment between the redesigned SAT’s research-based, empirical backbone and the Common Core.
Those “reports/analyses” were then used to persuade states such as Colorado and Michigan to drop the ACT in favor of the SAT, giving the College Board thousands of students in additional market share.
Of the many concerns raised by the Content Advisory Committee, here are the top three:
Item Quality: Committee members were very concerned with the quality of the items the College Board brought to committee meetings for review. Their biggest concern was the large number of items that were mathematically flawed; items that did not have correct answers; and items that did not have accurate or realistic contexts. Some members even went as far as stating that they had never seen so many seriously flawed items.
Development Schedule: Committee members felt that schedules did not allow them enough time to perform thorough reviews. Given the large number of items they had to review (and the poor quality of the items), they needed more time to provide meaningful comments and input.
Development Process: Committee members felt that the process used to develop the items was inadequate. They felt that the process lacked the rigor required to produce the high quality items necessary for item data to be useful. (https://www.linkedin.com/pulse/shining-spotlight-dark-corners-college-board-concerns-manuel-alfaro?trk=mp-reader-card)
Alfaro also indicates that an abnormally high number of items were revised, often to the point of being completely rewritten, after being pre-tested. As a result, some questions that appeared on the actual test had effectively never been vetted.
At least this explains why the College Board wouldn’t let tutors into the first administration of the new exam. The only reason to surround a test with that type of secrecy is to try to hide how poorly written the test is. If the College Board had so few items benchmarked for validity, it would also explain why the March test was reused in June.
Alfaro has also started a petition to ask the White House to investigate the College Board’s misdoings, but even if he does succeed in getting enough signatures, if I suspect that might be akin to asking the fox to check up on the henhouse. Coleman and the Common Core crew have deep ties to the Obama administration, via Arne Duncan. These are problems that go all the way to the top.
I agree that College Board has transparently been lobbying states aggressively to move into the ‘state test’ game. While many states offer either or both tests state-wide, only 12 have been using either the ACT or the SAT as part of their state and/or federal accountability measures. https://www.edweek.org/ew/section/multimedia/state-testing-an-interactive-breakdown-of-2015-16.html scroll down to point 4. The push in Colorado to replace the PARCC would make 13.
In April, The New Rules for the ESSA (replaced No Child Left Behind) were announced that pave the way for states to use the SAT or the ACT in place of state tests – in ways to meet the Fed. testing/accountability requirements. I believe this was an easy move based on years of positioning and lobbying, and a growing backlash both to the Common Core, and to over testing (the opt out movement) in general. https://www.washingtonpost.com/news/answer-sheet/wp/2016/04/28/new-u-s-rules-for-standardized-testing-have-been-drafted-heres-what-they-mean-for-kids/ However, there are still challenges around how to provide appropriate accommodations for students with disabilities which meets Federal standards – which are different than the procedures in place for either test.
Mr. Alfaro’s postings are very troubling. Lack of validity, lack of tests of reliability, and this will be used not only to sort students college options, but also to rate schools and effect the flow of dollars to schools (I believe that in some states higher school scores generates more $$)?
The behind the scenes peek given here indicates that the College Board rationale for it’s Redesign is transparently false. It isn’t about even out opportunities for students it is about raking in huge dollars (and at some point, at your indulgence, I would *love* to share my perspective as an urban educator who works in a school setting with predominantly minority and lower income students about how ridiculous the ‘Equity’ argument from College Board really is – both from a logical fallacy stand point, and from the actions taken.) Another case of wondering on what grounds College Boards is able to retain its status as a Non Profit organization.
Alfaro has gone quiet. I guess he got his hush money?
Or maybe the College Board “disappeared” him. (Kidding! Well, sort of…)
Alfaro is not the type to go quiet or accept hush money. He’s an honorable person when it comes to equity and fairness, particularly if it affects the poor and minority.
I urge folks to build momentum to expose this. He would not risk his reputation for being merely disgruntled.