Thursday, March 22, 2012

Neuroscience and psychology journal papers are like sausages and legislation.....

.....you don't want to see how they are made (believe me!)

"Of course, there is a wealth of exciting, valid, rigorous neuroimaging studies published, and the field is slowly becoming more standardised and robust as it matures. But, as I wrote in Twitter, the majority of neuroimaging studies I come across are so flawed, either due to design or statistical errors, that they add virtually nothing to my knowledge."

- neuroscientist Daniel Bor, from this much-recommended blog article with many interesting comments from other scientists:

Bor, Daniel (2012) The dilemma of weak neuroimaging papers. Daniel Bor author and neuroscientist. (blog) March 8th 2012.
http://www.danielbor.com/dilemma-weak-neuroimaging/#comments

Daniel Bor has particular interest to me because he is one of the many scientists who has studied Daniel Tammet, who has been the subject of much of my writing in the last few months. In the interesting and rather alarming discussions at Bor's new blog about the flaws of neuroscience neuroimaging journal papers (and other areas of science as well) there is some revealing discussion about the science journal Nature Neuroscience, the journal which published the "Routes to Remembering" study in 2002-2003, a study that has been much chewed-over at the blog in the last few months. Other science journals that have been cited in the past in my blog are also discussed. Bor has a new book to promote, which was probably the inspiration for the establishment of his blog.

Another quote, this one about the science of psychology:

"In this article, we accomplish two things. First, we show that despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not."

This quote is from the abstract of this journal paper from last year:

Simmons, Joseph P., Nelson, Leif D., Simonsohn, Uri (2011) False-Positive Psychology Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science. November 2011 vol. 22 no. 11 1359-1366. Published online before print October 17, 2011, doi: 10.1177/0956797611417632
http://pss.sagepub.com/content/22/11/1359.short

and of course, we've known for a long time that medical journals have been horribly corrupted by the influence of pharmaceutical companies:

Big Pharma: tricks of the trade. Counterpoint. ABC Radio National. 12 March 2012.
http://www.abc.net.au/radionational/programs/counterpoint/harriet/3871358

I think I'll just go read my horoscope and play with my magic healing crystals.....

No comments: