I Know What I Want to Know!

“We all make mistakes and, if you believe medical scholar John Ioannidis, scientists make more than their fair share. By his calculations, most published research findings are wrong.

“Dr. Ioannidis is an epidemiologist who studies research methods at the University of Ioannina School of Medicine in Greece and Tufts University in Medford, Mass. In a series of influential analytical reports, he has documented how, in thousands of peer-reviewed research papers published every year, there may be so much less than meets the eye.

“These flawed findings, for the most part, stem not from fraud or formal misconduct, but from more mundane misbehavior: miscalculation, poor study design or self-serving data analysis. ‘There is an increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims,’ Dr. Ioannidis said. ‘A new claim about a research finding is more likely to be false than true.’

“The hotter the field of research the more likely its published findings should be viewed skeptically, he determined.

“Take the discovery that the risk of disease may vary between men and women, depending on their genes. Studies have prominently reported such sex differences for hypertension, schizophrenia and multiple sclerosis, as well as lung cancer and heart attacks. In research published last month in the Journal of the American Medical Association, Dr. Ioannidis and his colleagues analyzed 432 published research claims concerning gender and genes. Upon closer scrutiny, almost none of them held up. Only one was replicated.

“Statistically speaking, science suffers from an excess of significance. Overeager researchers often tinker too much with the statistical variables of their analysis to coax any meaningful insight from their data sets. ‘People are messing around with the data to find anything that seems significant, to show they have found something that is new and unusual,’ Dr. Ioannidis said.

“In the U. S., research is a $55-billion-a-year enterprise that stakes its credibility on the reliability of evidence and the work of Dr. Ioannidis strikes a raw nerve. In fact, his 2005 essay ‘Why Most Published Research Findings Are False’ remains the most downloaded technical paper that the journal PLoS Medicine has ever published.

"’He has done systematic looks at the published literature and empirically shown us what we know deep inside our hearts,’ said Muin Khoury, director of the National Office of Public Health Genomics at the U.S. Centers for Disease Control and Prevention…

“Every new fact discovered through experiment represents a foothold in the unknown. In a wilderness of knowledge, it can be difficult to distinguish error from fraud, sloppiness from deception, eagerness from greed or, increasingly, scientific conviction from partisan passion. As scientific findings become fodder for political policy wars over matters from stem-cell research to global warming, even trivial errors and corrections can have larger consequences.”

(“Most Science Studies Appear to Be Tainted By Sloppy Analysis.” Robert Lee Hotz. Wall Street Journal: September 14, 2007. pg. B.1)

SO TORTURE THE DATA until they confess. There has to be a nugget of what I want to find somewhere in here! We all yearn for more and more of what we already know, not any more of what we don’t like to hear. (See Dylan below: “It'll soon shake your windows, and rattle your walls…” )