15 to 42 percent of medical research are false positives (Yet Another Calculation)

A while ago I found a very interesting paper from Leah R. Jager and Jeffrey T. Leek  via a post in the Simply Statistics blog arguing that most published medical research is true with a rate of false positives among reported results of 14% ± 1%.  Their paper came as a response to an essay from John P. A. Ioannidis and several others authors claiming that most published research findings are false.

After dealing with some criticisms Mr. Leek made a good point in his post:

“I also hope that by introducing a new estimator of the science-wise fdr we inspire more methodological development and that philosophical criticisms won’t prevent people from looking at the data in new ways.”

And thus, following this advice, I didn’t let criticisms prevent me from looking at the data in a new way. So for this problem I have devised a probability distribution for p-values to then fit the data via MLE and infer from there the rate of false positives.

pvalues PDF CDFSo this is my take; 15.33% rate of false positive with a worse case scenario of 41.75% depending on how mischievous researchers are but, in any case, and contrary to what others authors claim, most medical research seems to be true.

Continue reading

Climategate (1/3): be careful what you model for because you might get it

Deception is all around us, in every little parcel of our life; from our personal Bart Simpson’s “It wasn’t me” to our local TV news host selling us the latest “You’re not going to believe this” but we eventually do. One might just wish there would exist communities out there with higher standards like, for example, Christians priests but, nope, they cover up pedophile networks in order to preserve The Church’s “good” name. But how about the atheist priests a.k.a scientists? How about their standards?

Well, unfortunately the community of scientists might have more to do with priesthood than one might expect or desire, and a nice example of this would be the Climategate (or the Climatic Research Unit email controversy, as some people had the kindness to rename the Climategate article in the Wikipedia following Fox News’ motto “fair and balanced” )

So in this post I am going to replicate earlier studies on global warming to uncover how over pessimistic were the maths models of the past, but I will also talk about human weakness, and scientists are human… for now.

Mike's Trick
Click to watch Dr. Richard A. Muller (Professor of Physics at the University of California at Berkeley) talk about “Mike’s Trick” and the Climategate.

Continue reading