|
scooped again
Two Spanish researchers have shown (original here) that two leading journals routinely publish statistical errors: The analysis revealed that at least one error appeared in 38 per cent of the Nature papers and 25 per cent of the British Medical Journal papers looked at. Furthermore, the study estimates that four per cent of results reported to be statistically "significant" may not be significant after all.Yet again, the Spanish study is an example of someone actually doing something I thought of some time ago. (Fortunately for me, I'm usually only pleased when this happens, because I know perfectly well that I'll never do anything with the idea.) I am woefully ignorant of statistics, and probably have published overly simplistic analyses myself (though I am careful about claims of significance, and am confident that I've made no errors there). This sorry state is much more prevalent among biomed researchers than it ought to be, so I'm not suprised by the study's findings. Garcia-Berthou and Alcaraz also make another point upon which I've been known to wax shrewish: As well as warning researchers and editors to be more careful with data, they also urge the publication of raw data online. "If we had that, we could check the results," Garc�a-Berthou says. "Some journals already publish supplements online, but it's rare, and I think it should become commonplace."I think it should by now be viewed as low-rent not to make your raw data available online. There's no reason not to do it, unless you're hiding something; if the journal doesn't provide the option, the server space and bandwidth costs are well within reach of any research institution. I'm convinced that it will become a standard part of scientific publishing. (Obdisclosure: I haven't made any of my raw data available online, even though it was about the first thing I thought of when I came across the net, way back in 1993. I could never convince the higher-ups that it was a good idea. I'll start doing it as soon as I'm high enough on the food chain to insist on it, which I hope will be from the next paper onwards, paying for the hosting myself if need be.) Comments Reminds me of this, one off your list of favs if I remember correctly. :-) R
Here, in this little Bay,
A new Handbook of Data Analysis (published by Sage -- London) includes a very useful chapter on statistical significance. It includes cautions and examples of serious missteps. Post a comment |
RSS Feed
Links: (formerly Malice Aforethought) me spousal unit Bloglines account Simpy account Connotea account OpenWetWare userpage googlebombs for good Roe; Wade; Roe v Wade abortion Jew Seldovia Herald blogroll: Archives: March 2010 February 2010 January 2010 October 2009 July 2009 June 2009 May 2009 April 2009 March 2009 January 2009 December 2008 November 2008 October 2008 September 2008 August 2008 July 2008 May 2008 April 2008 March 2008 February 2008 January 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 June 2007 May 2007 April 2007 March 2007 January 2007 December 2006 November 2006 October 2006 September 2006 August 2006 July 2006 June 2006 May 2006 April 2006 March 2006 February 2006 January 2006 December 2005 November 2005 October 2005 September 2005 August 2005 July 2005 June 2005 May 2005 April 2005 March 2005 February 2005 January 2005 December 2004 November 2004 October 2004 September 2004 August 2004 July 2004 June 2004 May 2004 April 2004 March 2004 February 2004 January 2004 December 2003 |
You reminded me of a principle I learned in college statistics, many years ago. MINITAB, a statistics and number-crunching program, had just been made available, and we could now run all sorts of tests for significance. The professor warned us that if we had a data set of (say) 20 variables, and cross-tested each of them against each of the others, resulting in 190 cross-tests, using a 5% significance level, then we could expect 5% of 190, or 9 to 10, of the correlations to be significant, but that doesn't necessarily mean that anything's going on -- it would be surprising if we found no statistically significant correlations. He made the broader point, too often ignored in the real world, that statistical correlation does not imply causation or actual relationship.