ClimateGate…And “Stereotype ThreatGate”?
“Stereotype Threat”, the currently popular theory that
blacks average lower scores on cognitive tests in response
to stereotypes held by the testers, just a product of
publication bias?—defined as
“the tendency for researchers, editors, and pharmaceutical companies to
handle the reporting of experimental results that are
positive (i.e. they show a significant finding)
differently from results that are negative (i.e.
supported the null hypothesis) or inconclusive“.
whistle-blowing publication of emails from within the
Climate Research Unit
of East Anglia University
plotting the manipulation of research results
demonstrated again the unsurprising effects on scientists of
more of what you pay for!
hardly surprising that some climate scientists want climate
to be considered a very big deal.
Nor is it startling that, for example,
more medical journal articles tend to get published when a
drug company`s clinical trial of a
new drug turned out positively than when it had no
effect. After all, who paid for the trial?
about IQ research?
When you consider how our society`s reward
system is structured, what`s actually striking is
more corrupted by career considerations. Nothing is
worse for your future in academia than simply mentioning in
public the body of research on IQ. (Ask geneticist James D.
Conversely, nothing could be better for your career than to
come up with a politically-correct finding about IQ, the
most unpopular topic in the human sciences.
Yet, the facts about IQ tend to be so one-sided that the
anti-IQ case has been largely left to be argued by
lightweights, such as business book author
Malcolm Gladwell. (See his
New York Times
dispute with Harvard cognitive science heavyweight
Pinker, in which Gladwell
is left with little ammunition other than
including an attack on me).[
Let`s Go to the Tape,
Letter By Malcolm Gladwell, NYT< November 19, 2009]
Over the last 15 years, the most popular
IQ has been
The New York Times summarized it in its 2009 Year in Ideas featurette on
the purported “Obama
Effect“—the widespread assumption that the
politician`s success might
1995, two Stanford psychologists,
Claude Steele and Joshua Aronson, demonstrated that
African-American college students did worse on tests of
academic ability when they were exposed beforehand to
suggestions that they were being judged according to their
race. Steele and Aronson hypothesized that this effect,
which they labeled stereotype threat, might explain part of
the persistent achievement gap between white and black
students. In the years since, this idea has spread
throughout the social sciences.”
summarized the logic behind this extrapolation in my 2004
some point back in the mists of time, a stereotype somehow
emerged that blacks do less well on the SAT. So, now, blacks
are seized by panic over the possibility they might mess up
and score so poorly that they validate this stereotype. And,
indeed, this nervousness makes them score
exactly as badly as the stereotype predicted they would.”
Stereotype Threat is a beautiful theory. Indeed, I myself
have often felt there might even be a little bit of truth to
the idea that expectations matter—even though common sense
suggests that incentives matter more.
But now, it turns out that the vaunted evidence for this
wildly popular concept rests heavily upon another Effect,
Drawer Effect—defined as
“the practice of
researchers filing away studies with negative outcomes”.
We seem to have another
Climate Research Unit scandal on our hands.
researcher, who doesn`t want his name or any potentially
identifying information mentioned, for unfortunately obvious
career reasons, recently attended a presentation at a
scientific conference. Here is his summary of what he heard:
“One talk presented
a meta-analysis of stereotype threat. The presenter was able
to find a ton of unpublished studies.
conclusion is that stereotype threat does not exist. The
unpublished and published studies were compared on many
indices of quality, including sample size, and the only
variable predicting publication was whether a significant
effect of stereotype threat was found. …
“This is quite embarrassing for psychology as a science.”
Here`s the abstract of the presentation he
p. 68 of the PDF)
experiments have been conducted to show that African
Americans` cognitive test performance suffers under
stereotype threat, i.e., the fear of confirming negative
stereotypes concerning one`s group. A meta-analysis of 55
published and unpublished studies of this effect shows clear
signs of publication bias.”
Stereotype threat and the cognitive test performance of
by Jelte M. Wicherts & Cor de Haan University of Amsterdam
In other words, if a study doesn`t find
the existence of stereotype threat, it`s
less likely to see the light of day. Positive results
are more appealing to journal editors, and politically
correct positive results are loveliest of all. In contrast,
how much of a market is there for punching holes in
society`s fondest hopes?
Dutch researchers continue:
“The effect varies
widely across studies, and is generally small. Although
elite university undergraduates may underperform on
cognitive tests due to stereotype threat, this effect does
not generalize to non-adapted standardized tests,
high-stakes settings, and less academically gifted
“Stereotype Threat” mostly seems to exist in settings
where test-takers, such as
undergraduates”, are smart enough to pick up on
researchers` hints about what results they hope to publish.
In the marketing
research industry, it`s well known that survey respondents
respond with the answers that they surmise the pollster
wants. Human beings like to be cooperative if it doesn`t
cost them anything. Similarly, it`s likely not hard for
black students at top research universities to gather that
they can benefit their professor with a publishable paper
just by working less diligently.
doesn`t seem to exist where the test is important enough to
matter to the students.
not shocking either. One obvious problem with the notion
Threat” was proven was that it would be flagrantly
unethical for researchers to try to induce artificially
lower test scores among blacks, but not whites, on
high-stakes tests, such as the
Graduate Record Exam.
most of the published evidence for the syndrome comes from
meaningless tests concocted by the researchers—tests that
don`t matter much to students.
In fact, Claude Steele`s original study
found that he could get his black students to score
worse on his
made-up test than their SAT scores would predict, not that
he could lower their high-stakes tests, or get them to score
better on a meaningless one. In a 2004 article in the
Paul R. Sackett,
Chaitra M. Hardison, and Michael J. Cullen reported:
“Thus, rather than showing that eliminating threat
eliminates the large score gap on standardized tests, the
research actually shows something very different.
Specifically, absent stereotype threat, the African
American-White difference is just what one would expect
based on the African American-White difference in SAT
scores, whereas in the presence of stereotype threat, the
larger than would be expected based on the difference
in SAT scores.”[
Stereotype Threat as Accounting for African American–White
Differences on Cognitive Tests,
January, 2004 (PDF)]
In summary, Wicher`s and de Haan found:
cannot explain the difference in mean cognitive test
performance between African Americans and European
won`t embarrass, much less stop, the Race Deniers, of
course. But they will have to go back to the drawing board
on this one.