New, Probably Not Improved SAT Questions
04/16/2014
A+
|
a-
Print Friendly and PDF

Everybody complains about the SAT but nobody really explains why they are complaining (chief reason: their kids didn't get a perfect score), so "reforms" of the SAT traditionally flounder due to the general decadence of public thought in modern America. So, the latest changes in the SAT lack honest explication of what they intend to achieve.

As Herrnstein and Murray liked to point out, modern America is a rich place in part because we have standardized national tests in which small town boys like Murray and Jewish lads like Herrnstein could outshine the boarding school scions. America was particularly obsessed with finding talent for about a decade after Sputnik in 1957. But then along came civil rights and other obsessions, and the national clarity that was briefly achieved due to the fear of nuclear destruction has been eroded by wishful thinking and self-serving conniving.

We can hope that the College Board's Common Core wunderkind David Coleman knows what he's doing, but the whole topic of intelligence testing is so politically radioactive that we can't have an honest discussion in public of it.

From the NYT:

Revised SAT Won’t Include Obscure Vocabulary Words

By TAMAR LEWIN   APRIL 16, 2014

The College Board on Wednesday will release many details of its revised SAT, including sample questions and explanations of the research, goals and specifications behind them.

“We are committed to a clear and open SAT, and today is the first step in that commitment,” said Cyndie Schmeiser, the College Board’s chief of assessment, in a conference call on Monday, previewing the changes to be introduced in the spring of 2016.

She said the 211-page test specifications and supporting materials being shared publicly include “everything a student needs to know to walk into that test and not be surprised.”

Uh, why is it good that test-takers are not surprised? The surest way to make sure students aren't surprised is to release the test ahead of time. Would that be good? (In South Korea, moles within ETS apparently routinely release the upcoming tests to their test prep clients. Does that happen with the English language version in America?)

It's easy to yank Americans' chains with rhetoric that make lowbrow appeals to fairness. "It's just not fair that tests asks questions that students students don't already know the answers to!" Thinking through the implications of a particular methodology is hard, especially if you are disinterested, while emoting is easy.

One big change is in the vocabulary questions, which will no longer include obscure words.

I'm hoping that the College Board is worried that Tiger Mothers are forcing their children to memorize a couple of thousand SAT words. But it would seem like the way to fight gaming the system is to increase the number of SAT words, not decrease it. My vocabulary is generously estimated at 40,000 words, so it would hardly be absurd to have a pool of say, 10,000 words from which test questions are drawn.

Instead, the focus will be on what the College Board calls “high utility” words that appear in many contexts, in many disciplines — often with shifting meanings — and they will be tested in context. For example, a question based on a passage about an artist who “vacated” from a tradition of landscape painting, asks whether it would be better to substitute the word “evacuated,” “departed” or “retired,” or to leave the sentence unchanged.

Here's the sample question:

...As Kingman developed as a painter, his works were often compared to paintings by Chinese landscape artists dating back to CE 960, a time when a strong tradition of landscape painting emerged in Chinese art. Kingman, however, vacated from that tradition in a number of ways, most notably in that he chose to focus not on natural landscapes, such as mountains and rivers, but on cities....

A) no change

B) evacuated

C) departed

D) retired

So, that's less of a vocabulary question than it's a reading comprehension question or a writing style question. They've found a passage with a poorly chosen word and ask you to copy edit it by replacing it with a more obvious one. Which is fine. But ... big whoop. Both vocabulary and reading comprehension are both highly g-loaded. And that's good. 
But, if you've got two kinds of highly g-loaded questions, why throw one kind away and put all your eggs in the reading comprehension basket?
  
Another aspect of former McKinsey consultant David Coleman's changes are the McKinseyification of the SAT reading samples. I don't know if they can afford Malcolm Gladwell extracts, but one of the sample questions of vocabulary-into-reading-comprehension is from Dr. Vibrant, Richard Florida:
Question from Dr. Vibrant

Everybody talks about the importance of "critical thinking skills," but Richard Florida's entire career is testament to how well you can do for yourself in 21st Century America without any.

Honestly, being able to grok Richard Florida's articles on the Rise of the Creative Class probably does correlate highly with making money in modern America.

From the LA Times:

"This test will be more open and clear than any in our history," Cynthia B. Schmeiser, chief of assessment for the College Board, said during a media briefing. "It is more of an achievement test, anchored in what is important and needed for kids to be ready and succeed in college. The process used to define what is being measured is radically different than what we've used in the past and what is used in other tests."

The SAT is taken by about 1.7 million students annually but has been losing ground to the rival ACT, which is gaining nationwide acceptance after being more prevalent historically in the Midwest and the South. Many education experts view the ACT as more of an achievement exam, one in which students are less likely to be helped by coaching.

Does anybody know whether the SAT is more coachable than the ACT? My guess would be that, being traditionally the coastal test, while the ACT is the inland test, the big difference is that the SAT has more Asians taking it, which means it has more fanatical test prepping.

From the Washington Post:

For the College Board, the new SAT could help end the lingering public perception that the test is about IQ or aptitude. Previous revisions ditched analogies and antonyms — portions of the old verbal test seen as tricky and unrelated to what schools teach. Making the SAT more of an achievement test, one analyst said, could be a boon for students who stress about test preparation.

Nobody is even trying to make sense anymore.

We actually know how to make testing substantively more accurate: stop having paper and pencil tests and put them instead on a computer and have the computer change how hard the questions are based on how the test-taker is doing. A lot of major tests have shifted to that, such as the military's ASVAB back at the end of the 20th Century. But the College Board doesn't want to spend the money to do that yet.

Print Friendly and PDF