I have spent much of the past two years analyzing and reconstructing some of the basic studies used by the Intergovernmental Panel on Climate Change (IPCC) to support their conclusions about global warming and in turn to promote policies on climate change.
My conclusion, in three peer-reviewed publications that Natuurwetenschap & Tachniek, the National Post, and the Wall Street Journal have recently reported on, is that the major science journals and organizations lag far behind the corporate world in ensuring the quality of researchers’ data.
In the corporate world, there is simply no question about the importance of providing audit trails, and while they can take many different forms, they all serve the purpose of ensuring the validity of information used for investment decisions.
Flawed “Hockey Stick” Data
The 2001 IPCC report produced findings that have guided investment decisions, which vastly exceed the sums involved in even the largest financial scandals of recent years. Since the IPCC leaned heavily on a novel approach called a “multiproxy climate study” and in particular the “hockey stick graph” of Mann et al. that purported to show extraordinary climate change, this is where I’ve focused my attention. (Editor’s note: See “Nature Admits Widely Cited Global Warming Graph Was Erroneous,” Environment & Climate News, September 2004.)
An audit trail in this case is easily defined: the data in the form used by the authors and the computer scripts used to generate the results. In principle, these can be easily buttoned up and publicly archived.
Yet none of the major multiproxy studies has anything remotely resembling a complete due diligence package, and most have none at all. The author of one of the most quoted studies (Crowley and Lowery, 2000) told me he has “misplaced” his data. In the case of the Mann et al. (1998, 1999) study, used for the IPCC’s “hockey stick” graph, Mann initially claimed to be unable to remember where the data was located, then provided inaccurate data, then provided a new version of the data which was inconsistent with previously published material, etc.
Refusal to Disclose Data
Authors typically refuse to make their source code and data available for verification, even with a specific request. Even after inaccuracies in a major study had been proven, when we sought source code, the original journal (Nature) and the original funding agency (the U.S. National Science Foundation) refused to intervene.
In the opinion of the latter, the code is Mann’s personal commercial property. Mann recently told the Wall Street Journal that “giving them the algorithm would be giving in to the intimidation tactics that these people employ.” My first request for source code was a very simple request and could in no way be construed as “intimidation.” The issue neatly illustrates the widespread refusal to make public the underlying data for these influential studies.
Cursory Peer Audits
IPCC proponents place great emphasis on the merit of articles that have been “peer reviewed.” However, peer review for climate publications, even by eminent journals such as Nature or Science, is typically a quick, unpaid read by two or three knowledgeable persons, usually close colleagues of the author. It is unheard of for a peer reviewer to actually check the data and calculations.
In 2004, I was asked by a journal (Climatic Change) to peer review an article. I asked to see the source code and supporting calculations. The editor said no one had ever asked for such things in 28 years of his editing the journal. There is nothing at the journal peer review stage in climate publications that is remotely like an audit.
Although the IPCC and similar agencies have many committees and meetings (usually in nice places), they do not carry out any audit or verification activities.
While insiders have long known this, it was recently admitted in written answers by the author of the hockey stick study (Michael Mann) to the U.S. Senate in the fall of 2003. “It is distinctly against the mission of the IPCC to ‘carry out independent programs,'” Mann wrote. Thus, if a paper has passed the cursory journal peer review process, there may not be any subsequent hurdles prior to adoption by the IPCC.
Serious Error in Calculations
Through my own checking, I found that the calculations behind the most famous IPCC graph–the 1,000-year climate hockey stick–contained a serious calculation error that invalidates the results. In this case, the methodology had been inaccurately described in the journal publication.
I also found an influential but unreported alteration to a key data series, where the alteration had been disguised by a (perhaps unintentional) misrepresentation of the start date of the underlying data.
The math involved is not particularly sophisticated: The errors would have been discovered long ago had there been even routine checking.
For all the billions of dollars being spent on the climate change industry, and the thousands of people working full time on this issue, it was nobody’s job to check whether the IPCC’s main piece of evidence was right.
Insufficient Self Audits
IPCC’s inattentiveness to verification is exacerbated by the lack of independence between authors with strong vested interests in previously published intellectual positions and IPCC section authors. For example, Mann had published an academic article announcing the 1990s were the warmest decade in human history. He then became IPCC section author for the critical section surveying climate history of the last millennium, adopting the very graph used in his own paper.
Corporate prospectuses, by contrast, require qualifying reports from independent analysts. In business, “full, true, and plain disclosure” is a control on stock promoters. Although not always successful, it gives an enforcement mechanism. There is no such standard in climate science.
In the Mann study, for example, there were important and pertinent adverse results, known to the authors, which were not reported. In fairness, the journals do not require authors to warrant full, true, and plain disclosure, and there is little guidance to authors as to what is and is not required. I’ve found that scientists strongly resent any attempt to verify their results. A typical reaction is: Don’t check our studies, do your own study.
I don’t think that businesses like being checked either, but it is one of the preconditions of being allowed to operate. Many of the most highly paid professionals in our society–securities lawyers, auditors–earn much of their income simply by verifying other people’s results.
Necessity for Transparency
Businesses developed checks and balances because other peoples’ money was involved, not because businessmen are more virtuous than academics. Back when paleoclimate research had little importance outside academic seminar rooms, the lack of any adequate control procedures probably didn’t matter much.
However, now that huge public policy decisions are based, at least in part, on such studies, sophisticated procedural controls ought to be developed and imposed. Climate scientists cannot expect to benefit from public money and influence public policy without accepting the responsibility of providing much more adequate disclosure and due diligence.
Steve McIntyre is a mineral exploration businsessman and co-author, with Ross McKitrick, of Hockey Sticks, Principal Components, and Spurious Significance published in Geophysical Research Letters, copyright 2005, American Geophysical Union (doi: 2004GL012750). This essay was published in the February 15 Financial Post (http://www.canada.com/national/nationalpost/financialpost/index.html) and is used with permission.