Earlier in the summer we looked at a live controversy about a social science paper making claims about the impact of gay parents on the well being of children. Controversy over the paper drove the University of Texas to undertake an investigation of whether scientific misconduct was involved.
The investigation is complete, and finds no evidence of misconduct (falsification of data or other unethical practices). Meanwhile the journal that originally published the article has conducted an 'audit' of it and they find it seriously flawed.
Careful review and extensive discussion of this paper emerged because of the politically charged nature of the conclusions. This paper was approved through a peer review process for publication, as are thousands of others every week. It has real flaws, uncovered in the more intense review it received. How many of those other papers share flaws at this scale?
Early in the summer, Austin posted a link to a paper concluding that most research studies are wrong.
Again, this paper is not claiming misconduct, it's just reminding us that the standard for discovery (a 1 in 20 probability of chance occurrence), combined with the strong bias against publishing null results, guarantees that most interesting new discoveries will in fact be wrong. There's another paper on this effect in this month's American Scientist magazine.
All of this calls for a strong dose of humility in research. Initial positive findings ought to be treated as a reason for interest, rather than as discoveries.
No comments:
Post a Comment