environmentalresearchweb blog

« 2050: an all-electric future? | Main | Renewable breeders »

Settled science with a vengeance

One of the more unseemly sideshows of the Climategate fuss has been the argument about editorial treatment of two papers in the International Journal of Climatology. In late 2007, Douglass, Pearson, Christy and Singer published, online, a paper about the mismatch between modelled and observed temperature trends in the lower atmosphere. The paper did not appear in print until October 2008, so I will call it D08. It came just before a paper, S08, by Santer and co-authors that criticized the Douglass paper.

You can find more than you may want to know about the editorial treatment of D08, as filtered through the minds of Douglass and Christy, in a blog contributed in December 2009 to the right-wing magazine American Thinker. But that is not what I want to discuss here. I am more interested in the golden opportunity offered by American Thinker, 14 months on, for Douglass and his co-authors to rebut the criticisms of S08. Golden as it was, they passed it up.

The core of the argument is the assertion in S08 that D08 used an incorrect statistical test. Although the connection to glaciers may seem tenuous, it is real and of broad interest: the dispute is about wiggle room. I know I said wiggle room was dull — but just look at how excited we get about it. This kind of thing is the essence of the search for expensive signals buried in distracting noise.

The aim of the test used in D08 is to decide whether two sets of numbers, in this case observed temperatures and modelled temperatures, are “different” in a sense that can be defined precisely and with a known amount of confidence. There comes a point during the test where you have to divide by the square root of a number called the “effective sample size”. In general this number is smaller than the sample size, because correlations between the numbers in the sample reduce the amount of wiggle room you have while making your test decision. In the jargon of statistics, the wiggle room is called the “degrees of freedom”.

If you use the sample size instead of the effective sample size, you get an insidiously wrong answer. Your error bars come out too small and you end up being too confident about your decision. This is precisely the trap into which D08 walked. S08 did the test properly, and concluded correctly that there is no reason to believe that, on average, the climate models are mis-modelling the observed temperatures.

The trap is not widely understood, even among scientists, but that is no excuse when, as did D08, you choose to play for high stakes. American Thinker gave them a chance to respond to the criticisms of S08, and all they produced was whingeing about the editorial process.

In an appendix to their blog, D08 offer a scientific discussion of their work. They say that S08 “strongly objected to the narrowness of our error bars. Their view was to allow models to have a very wide range of possibilities of trends (roughly the range from the coolest model to the warmest) no matter what their associated surface trends might be.” Never mind the “roughly”; the parenthesis misrepresents the meaning and importance of wiggle room. The bit about surface trends is a red herring. The quotation, and the appendix as a whole, show that D08 still misunderstand S08 comprehensively.

Santer and his co-authors are right. Douglass, Pearson, Christy and Singer are wrong. These two points ought to be at the centre of this part of the climate-wars discussion. They cannot be stressed too often.

There is a further irony in this story of denialism and wiggle room. An important part of the denialists’ weaponry is the term “settled science”, repeated over and over again as a criticism of the conventional wisdom about the climate. Yet the settled science elucidated in S08 features enormous error bars. The denialists often leave out the error bars, but D08 did have error bars. The trouble is that they were tiny, and wrong. Settled science with a vengeance.

Posted by Graham Cogley on May 3, 2010 3:00 PM | Permalink