08 February 2016

Will the new SAT be written by computer?

A colleague recently forward the following job ad to me — you can make of it what you will.

One does, however, have to wonder just who (or what) will be “meticulously writing, re-writing, measuring, testing, and balancing new questions before they are ever presented to students.” At what point will actual humans be involved? 

I guess we now know why the experimental section on the SAT could be safely jettisoned. When you can run high-powered simulations (gathered from Khan Academy student data?), who needs test-takers? 

And who needs ETS when there’s the Hive? 

DevOps Engineer

DevOps Engineer

Job ID: AD20152008-19605


The College Board, a national educational organization, is conducting a search for a Software Engineer II (DevOps Engineer)  in our Information Technology organization, who will be resident in our Reston, VA office.
Position Summary
We are building The College Board’s next-generation item authoring platform, called Hive.  Questions on the SAT, PSAT, and Accuplacer assessments are meticulously written, re-written, measured, tested, and balanced before they are ever presented to students.  Hive will manage all of that, in addition to providing versioning, workflow, and test assembly features. This is a mission critical application for The College Board.  Hive is also a brand new capability for us, which provides us a large degree of freedom to experiment and evolve our software development processes.
We are a small team of passionate engineers who love what we do.  We continuously improve the way we work and push the boundaries of our technology platform.
As a DevOps engineer at The College Board, you will:
  • Build and own our continuous delivery pipeline
  • Work closely with the development team to get new features in front of our customers
  • Learn, improve, and push the boundaries of our technology stack
  • Build automated testing to validate environment configuration and software deployments
  • Take ownership of development tasks and see them through to delivery
  • Coach other developers on Continuous Delivery
  • Bachelor’s Degree in Computer Science or a related field of study
  • Minimum of Four years of directly related experience
  • Experience with the following technologies:
o    C#, ASP.Net MVC, ServiceStack, Javascript, Knockout.js, SqlServer, EntityFramework
o    Amazon AWS: EC2, Cloud Formation, VPC, RDS, CodeDeploy
o    Jira, Confluence, Bamboo, Stash (Git), Svn
o    Powershell, Python, Bash
o    Windows, Linux
  • Experience with agile development methodologies
  • Excellent organizational, prioritization and time management skills
Additional Requirements
  • Experience setting up and managing automated build and deployment systems
  • Experience with Chef, Puppet, Salt, or Ansible
  • Familiarity with CloudFormation and JSON
  • Familiarity with system monitoring tooling like Nagios, SiteScope, Cloudwatch, etc.
  • Experience working in a Continuous Delivery environment
  • Team player that will cross discipline boundaries to help their peers
  • A pragmatic approach to development, and problem solving in general
  • Humble but motivated individuals
This position will be subject to a background check.
If you are interested in joining the College Board and you meet our qualifications, we encourage you to APPLY by following the link below to send us your resume and a brief introductory cover letter that includes salary expectations.
The College Board’s mission is to connect students to college success and opportunity. We are a not-for-profit membership organization committed to excellence and equity in education. Among our best-known products are the SAT®, PSAT/NMSQT®, and Advanced Placement Program®.


4 Responses

  1. Dan Lipford

    I’m not at all sure that computers couldn’t do a better job than some of the folks they’ve got writing their questions and answers now.

    The following text will soon appear in our blog post on the New SAT’s Writing and Language Test:

    “According to the CB, the New SAT’s Writing and Language Test asks students to be editors “and improve passages that were written especially for the test” [remember that for later]…

    “The transportation planner’s job might involve conducting a traffic count to determine the daily number of vehicles traveling on the road to the new factory. If analysis of the traffic count indicates that there is more traffic than the 6 current road as it is designed at this time can efficiently accommodate, the transportation planner might recommend widening the road to add another lane.”

    6 Select an answer
    B current design of the road right now
    C road as it is now currently designed
    D current design of the road”

    And this is where the “passages that were written especially for the test” comes into play and our faith in the ability of the CB’s passage and question writers is sorely tested. Sure, we can pick out the best of the lousy lot of answer choices that the CB provides, but the single best choice — “current road” — is notable for its absence…and this is a product of the people we’re being asked to trust to measure our students’ acumen with the English language. Hmmm.

  2. Charles LEE

    Reading Passages of GRE by ETS have been selected by a software called “source finder” for several years. It is not a surprise that CB finally seek aids from computer science to lower its cost of items writing.

  3. F. Smith

    From the looks of that job posting ad it does not appear as though Hive is actually writing the material for the SAT. Rather, HIVE is the platform that the College Board employees will use to write/organize/edit/analyze items and other data. In CS the word “platform” just means a tool that people use to accomplish tasks such as “versioning, workflow, and test assembly features.” Think of it like this: Facebook is a platform for social networking. The Facebook platform provides an interface and design to engage in social networking, but Facebook doesn’t say create your photos or write your profile for you. Pretty much every other company that writes standardized test also uses some sort of platform, so this isn’t something that is unique to the College Board.

    1. admin

      I understand that Hive is simply a management platform… for now. My point was that based on the College Board’s recent behavior and too-close-for-comfort ties to the tech industry (Khan Academy receives significant funding from the Gates Foundation, which bankrolled Common Core and effectively got Coleman installed as head honcho at the College Board), it wouldn’t surprise me in the least if the CB did at some point decide actually try having the test written by computer. It would certainly cut down on development costs.

Leave a Reply