Manuel Alfaro, a former executive director at the College Board, has written a series of posts on LinkedIn detailing the myriad problems plaguing the development of the new exam.
According to Alfaro, not only were many of the items developed for the first administration of the test extraordinarily problematic (see below), but many of the items that appeared on the test were not actually reviewed by the Content Advisory Committee until after the test forms had been constructed.
Committee members repeatedly attempted to call David Coleman’s attention to the problem, but were ignored.
Alfaro was also responsible for rewriting and rubber-stamping the redesigned test specifications in order to hide the fact that they were taken directly from the Common Core middle- and high school standards. (Because, of course, the College Board expected everyone to somehow forget that David Coleman, the head of the College Board, was also responsible for Common Core.)
The College Board completed the backstory for the test specifications by citing reports/analyses performed by independent groups as evidence of the alignment between the redesigned SAT’s research-based, empirical backbone and the Common Core.
Those “reports/analyses” were then used to persuade states such as Colorado and Michigan to drop the ACT in favor of the SAT, giving the College Board thousands of students in additional market share.
Of the many concerns raised by the Content Advisory Committee, here are the top three:
Item Quality: Committee members were very concerned with the quality of the items the College Board brought to committee meetings for review. Their biggest concern was the large number of items that were mathematically flawed; items that did not have correct answers; and items that did not have accurate or realistic contexts. Some members even went as far as stating that they had never seen so many seriously flawed items.
Development Schedule: Committee members felt that schedules did not allow them enough time to perform thorough reviews. Given the large number of items they had to review (and the poor quality of the items), they needed more time to provide meaningful comments and input.
Development Process: Committee members felt that the process used to develop the items was inadequate. They felt that the process lacked the rigor required to produce the high quality items necessary for item data to be useful. (https://www.linkedin.com/pulse/shining-spotlight-dark-corners-college-board-concerns-manuel-alfaro?trk=mp-reader-card)
Alfaro also indicates that an abnormally high number of items were revised, often to the point of being completely rewritten, after being pre-tested. As a result, some questions that appeared on the actual test had effectively never been vetted.
At least this explains why the College Board wouldn’t let tutors into the first administration of the new exam. The only reason to surround a test with that type of secrecy is to try to hide how poorly written the test is. If the College Board had so few items benchmarked for validity, it would also explain why the March test was reused in June.
Alfaro has also started a petition to ask the White House to investigate the College Board’s misdoings, but even if he does succeed in getting enough signatures, if I suspect that might be akin to asking the fox to check up on the henhouse. Coleman and the Common Core crew have deep ties to the Obama administration, via Arne Duncan. These are problems that go all the way to the top.