Thomas Sowell has a three part series on what “Studies prove…” in his syndicated column. [I, II, III] His experience with this is interesting, in light of, for example, the Pew Study discussed below. Sowell writes
My late mentor, Nobel Prize-winning economist George Stigler, used to say that it could be very instructive to spend a few hours in a library checking up on studies that had been cited. When I began doing that, I found it not only instructive but disillusioning.
A footnote in a textbook on labor economics cited six studies to back up a conclusion it reached. But, after I went to the library and looked at those six studies, it turned out that they each cited some other study — the same other study in all six cases.
Now that the six studies had shrunk to one, I got that one study — and found that it was a study of a very different situation from the one discussed in the labor economics textbook.
Then there was the one about how immigration decreases crime, and the Freakonomics scandal. (I don`t even count the Bellesiles thing; he was just flat-out lying, which is too crude for the “studies prove” people.)
Later Sowell writes
Once a minister was explaining to me the structure of his funeral orations. He said, “At this point, you are expected to say something good about the deceased. Now, Tom, if I were preaching your funeral, what would I say good about you at that point?”
He thought and thought — for an embarrassingly long time. Finally, he said gravely: “In his research, he always used original sources.”
I`ll take that.