September 15, 2006, 16:27
shufitz"publication bias"
From today's Wall Street Journal:
In ancient Greece, sailors who survived shipwrecks had their portraits displayed in a temple on Samothrace as a testament to the power of Neptune. When Diagoras of Melos was told that this proved that the gods insert themselves into the lives of men, he answered, "but where are they painted that are drowned?"
Today, showing only the rescued sailors would be called
publication bias, the tendency of scientists to report findings that support some point (Neptune rescues sailors) but to bury examples (drowned sailors) that undercut it. It has existed for years, most seriously in the failure to publish studies that cast doubt on the safety or efficacy of new drugs.
… journals are biased in favor of positive studies … "Positive" means those showing that some intervention had an effect, that some gene is linked to a disease – or, more broadly, that one thing is connected to another in a way that can't be explained by random chance. A 1999 analysis found that the percentage of positive studies in some fields routinely tops 90%. That is statistically implausible, suggesting that negative results are being deep-sixed. As a result, "what we read in the journals may bear only the slightest resemblance" to reality.
September 15, 2006, 17:00
dalehilemanOn a slightly different but closely related topic, acupuncture was considered a heathen "science" and Consumers Reports ridiculed it
But when Nixon suddenly decided for no apparent reason that China should be our friend, at once acupuncture was ok and CR virtually praised it to the skies, while acupuncture clinics popped up everywhere
Thus is science politicized
September 15, 2006, 18:13
KallehThat surely is the case in getting research published. It is
much harder to get something published when the independent variable shows no effect, rather than when there is a significant effect. But of course researchers know that, so they perform every kind of analysis available to find something to report.
Still, it has always seemed like more of a game to me than science.
September 15, 2006, 21:53
SeanahanI don't see how publication bias is anything other than a specific instance of selection bias.