A century ago or so, each college had its own admissions tests, but the inefficiencies of that became apparent. Eventually the country came up with two competing college admissions tests — the SAT and the the ACT — with colleges free to pile on eccentric essay questions or whatever they feel like. A few colleges may still have their own unique entrance exam (All Souls College at Oxford has a famous one), but in the U.S. customized admissions tests are quite rare.

Now, you might think that if national tests are good enough for Yale, they ought to be good enough for, say, the New Haven Fire Department. And there are companies that publish tests for fire departments across the country to use. Yet, as we saw with the 2009 Ricci Supreme Court case, nobody thought it odd that New Haven had spent $100,000 on a consulting firm to dream up a customized test just for New Haven. That's pretty common in the fraught world of hiring firemen.

In fact, much of the defense against Mr. Ricci's reverse discrimination lawsuit consisted of the allegation that the test wasn't customized enough: the consulting firm had borrowed a question from earlier tests that referred to "downtown" when there is no downtown in New Haven (or something like that — all I remember is that the lack of local customization of one question was a big deal in the national press for a couple of weeks in 2009.)

How, exactly, is the New Haven FD so different from, say, the New Canaan FD that New Haven needs to spend one hundred grand on its own test? Caltech, Yeshiva, the Air Force Academy, and Smith are probably more different from each other than most municipal fire departments are different from each other, and yet they all find the SAT and ACT helpful.

Now there are some nationally available general purpose job hiring tests like the Wonderlic IQ test famously used by the NFL. Still, a glance at the 17,000 words of federal guidelines on whether or not the EEOC will come down on you like a ton of bricks if you use formal testing as part of the hiring process demonstrates clearly that the answer to any legal question involving testing and hiring is Maybe.

If you started your own college and then, as it got more successful, you announced that you were going to mandate the SAT and/or ACT as part of the admissions process, nobody would blink an eye. But if you start a business that grows big enough to get on the EEOC's radar, can you assume that you can just use some battle-tested national test? Or do you have to validate that the national test specifically works at your not-so-unique company? 

Maybe.

Over the last century, industrial/organizational psychologists have put a lot of effort into understanding the mysteries of testing. One of their major findings is that you don't need all that many different tests. The kind of things that can be measured well by testing aren't very unique to one college or one company or one job or whatever. Test optimization runs into pretty severe diminishing returns.

And yet, almost nobody outside of the profession is aware of this major discovery of 20th Century social science. Why not? One reason is because a large number of the people who understand this — professional I/O psychologists — are employed to come up with new tests that will do what 100 years of I/O research says can't be done. 

Hey, it's a living.