The Lecture given in The Edinburgh Room at The Brisbane Club
on 1 May 2008
The history of science is replete with error and fraud.
Environmental science is no exception. Indeed, this area of science provides a hyperabundance of examples, thanks to the presence of two factors: a good cause and extensive reliance upon modelling, especially that involving sophisticated computer models.
The good cause -- one that most of us support -- can all too readily corrupt the conduct of science, especially science informing public policy, because we prefer answers that support our political preferences, and find science that challenges them less comfortable.
We would all wish to preserve the spiralled-horned ox, Pseudonovibos spiralis, because it is on the Red List of endangered species. Problem is, is doesn't seem to have existed in the first place.
And we might not have minded the apparent planting by US Federal Fish and Wildlife Department officers of fur from endangered Canadian lynx in Wenatchee and Gifford Pinchot National Forests in the Pacific Northwest in 2002.
When found out, the officials claimed that they were merely trying to test the reliability of testing methods, by covertly seeing whether the testing laboratories could identify real lynx fur if not told in advance. Critics suspected the samples had been planted in an effort to protect the national forests from logging, mining and recreation. The Executive Director of the Forest Service Employees for Environmental Ethics termed this response "a witch hunt in search of a false conspiracy".
This Executive Director, Andy Stahl, had what is known in policing circles as "form". In the 1980s, during the controversy over the logging in the Pacific Northwest, Stahl was involved in sponsoring the production of peer-reviewed science to support the Spotted Owl campaign to reduce old-growth logging.
Stahl put mathematical modelling entymologist Russell Lande in touch with scholars who suppled the data, and then helped find reviewers to produce a peer-reviewed publication. This was necessary because the only "science" then available on the spotted owl was an incomplete doctoral dissertation.
The Lande paper was created to suit the political campaign and was used together with the notion of precaution to win the day. Whereas it assumed an owl population of 2,500 and further assumed that logging old-growth forest would cause its extinction, subsequent research showed the species was far more numerous and, if anything, preferred regrowth forest. Regrowth forest provided more prey and more conducive hunting conditions than old-growth forest.
Remarkably, the leading journal Nature editorialised in support of those who had faked the Canadian Lynx evidence -- which tells us something about scientific journals.
The combination of the precautionary principle with endangered species legislation is a particularly seductive one, but it is the use of models into which value-laden assumptions can be smuggled that is particularly pernicious -- as a recent Australian example shows.
A case involving the Orange-Bellied Parrot in 2006 saw the merest hint of a parrot, together with some mathematical modelling (and the precautionary principle) used by the then Australian Commonwealth Environment Minister to disallow the construction of a wind farm that was environmentalists' preferred response to climate change, but was opposed by residents in a marginal Coalition government constituency.
Modelling for the Bald Hills wind farm on the Orange-bellied Parrot assumed the birds spent time at most of the sites of wind farms in Victoria, despite the fact that the birds had not been recorded at 20 of the 23 sites along the coast of Victoria, and despite active searches having been conducted. Only one or two sightings had been made at the other three sites.
The authors then assumed that the birds would remain present within a single wind farm location for six months -- the longest possible period the migratory species could remain at a winter site, and longer than any bird had been recorded at any site. They also assumed the parrot would make two passes through the Bald Hills site. They did all this to err on the side of caution.
So, while no parrot had been sighted within 50 kilometres of the proposed site, the minister then acted in accordance with the precautionary principle (and an election promise) to block Bald Hills on the basis of cumulative impact -- compounding the precaution already embedded in the assumptions underlying the modeling.
I have proposed in what I call Wood's Law that sightings of endangered species are clustered around the sites of proposed developments. This reflects not just the cynical uses of endangered species for political purposes, but partly also the fact that research for environmental assessments frequently finds species because the site has never previously been surveyed.
This "noble cause" corruption of science -- named for the "framing" by police of suspects "known" to be guilty is helped not just by the virtuous cause, but by the virtual nature of both the science and the context within which it occurs. Both conservation biology and climate science rely on virtual science. The former has seen people in check shirts counting deer scat give way to physicists and mathematicians, while the latter (unlike more traditional meteorology) has always involved more computing than fieldwork.
James Hansen, of NASA's Goddard Institute, for example, wrote his doctoral thesis on the climate of Venus, and -- contrary to what some of his critics might think -- it's clear he has never visited another planet. Computer models fed by scenarios based on economic models are the norm in climate science, and when we are dealing with climate impacts on biodiversity, we are often dealing with species-area modelling fed by the modelled results of the impact of climate models on vegetation.
It is important to understand the way in which the revolution in information technology has transformed the conduct of science. Its impact has come not just in the ability to model complex phenomena of which scientists a decade or so ago could only dream -- though that is part of the problem. Computer models are always subject to the Garbage In -- Garbage Out problem and they can never be a substitute for hypotheses tested against the cold, hard light of observational data.
Many of the scientists working with models appear to have forgotten that science is about testing predictions against data. They seem to have fallen victim to the trap long-recognised at IBM, where it used to be said that simulation was like self-stimulation: if one practised it too often, one began to confuse it for the real thing.
One problem with observational data in areas like climate science is that they themselves are subject to substantial massaging by computers before they are of any use. Even data collection, therefore, provides opportunities for subjective assumptions to intrude into the adjustments made to data to make them useful.
This highlights the importance of quality assurance processes, and there are no greater guarantors of quality assurance in science than contestation and transparency -- full disclosure of data properly archived and of methods, including computer code.
Society deems this fundamentally important when we are dealing with science such as drug trials, which are conducted under fully transparent conditions, ideally with separate teams making up doses, administering them, diagnosing effects and analysing data. We insist on regulatory guidelines, and we audit laboratories. We know that even when researchers are fastidious in pursuing impartiality, subjective assumptions can find their way into what become "data".
There are similar requirements imposed by stock exchanges for data such as core samples relating to mineral resources. Standards govern the collection, archiving and analysis of data. In Australia, these are laid down as standards by JORC -- the Joint Ore Reserves Committee. Even then, mistakes occur and there are consequences: shareholder value is destroyed or created.
In areas such as climate science we have made no similar demands. Data are routinely gathered, manipulated and modelled by the same research teams and the discipline has not insisted on anything like full transparency. Many of the people engaging in this science are then acting as advocates for particular policy responses. James Hansen is perhaps the most notable example in this regard, but there are numerous others, such as Stephen Schneider at Stanford.
The Intergovernmental Panel on Climate Change then allows the same people to act as lead authors, sitting in judgment on their own work and that of those who might differ with them. This corrupts the scientific process.
The work of former mining industry analyst Steve McIntyre in exposing the debacle of the Hockey Stick controversy in climate science and in finding that Hansen's computer generation of mean temperatures for the US had a Y2K problem (that meant that the hottest year shifted conveniently from the 1930s to the 1990s) are good examples of what is needed. But it is significant that these necessary correctives came from outside the climate science community.
The shift of the "warmest year" in the US was in itself a small change in the totality of climate science. But most of the mistakes tend to be in one direction, and that is in a politically convenient one. This underscores my point about the need for openness, transparency and sceptical challenging of science, especially where data collection, data preparation, data adjustment, modelling and interpretation all take place in the one institution.
Again, it is noteworthy that an amateur scientist, Anthony Watts, is responsible for a web-based audit of sites that generate data for that record, and he and his "citizen auditers" have found many sites that are likely to have produced a recent warming trend through poor siting or site maintenance.
It is worth reporting that Watts visited NOAA recently, and not only was he given a warm reception, but he found that the walls of the offices of those responsible for maintaining temperature records were covered with photographs of the stations he and his supporters have photographed. NOAA is grateful for the work they have done (at no cost to it), and the result is likely to be better data in the future. But its surface records continue to be based on flawed instrumentation that is subject to adjustment and compilation.
I would suggest that the need for sceptical auditing is even greater when the senior spokesman for the institution concerned is also a vociferous advocate for a particular policy position. James Hansen claims to have been muzzled by the Bush administration -- though Republicans were unkind enough to point to the 1400 or so media interviews he seems to have managed, and he managed to throw off the muzzle for long enough to endorse John Kerry in 2004.
The point about all this is that, while Michael Crichton once famously observed that "data is not Democrat or Republican, it's just data", we need to ensure we have institutions that prevent data from acquiring partisan characteristics.
Steve McIntyre was aware of the case of Bre-X, where gold assays were fabricated and now applies his considerable skills to auditing climate science -- to our enormous collective benefit. The proposition with climate change policy is that we are being asked to make substantial social investments in an enterprise that does not have the standards of transparency and accountability stock exchanges insist upon to prevent Bre-X situations, nor situations where subjective beliefs have intruded into analyses.
But to return to the impact of IT on all of this, we must recognise how the IT revolution has also revolutionised both the conduct of science and the way in which it is interpreted -- the way in which it enters politics and the policy process.
One of the impacts has been on peer review, the cornerstone of quality assurance in science. Publication after anonymous peer review in quality journals does not guarantee that the science is accurate, but it helps guard against inaccuracy.
Some journals in which key pieces of climate science are published do not maintain the standards of strict double blind refereeing that we take for granted in the social sciences. Geoscientists I raised this with thought that this would inhibit debate between authors and reviewers that might lead to fresh insights. Perhaps -- but if society is to take such science seriously, such conversations have to be secondary to quality assurance. We are well past Victorian gentlemen discussing interesting fossils they have found.
That problem aside, the internet has made it much more likely that the identity of an author can be tracked down, breaking down the anonymity that focuses reviewers on the quality of the reason and evidence presented in the paper.
Indeed, the internet has made possible increased international collaboration among scientists, while the increasing specialisation of knowledge has narrowed the circle of likely referees. Not only does the internet (and cheap air travel) increase the likelihood that authors are known to potential referees, it increases the likelihood that they have worked together. The IPCC has assisted this process, by engaging many of them on a common task and producing that enemy of all good science, a consensus.
Edward Wegman performed a social network analysis of those working on multiproxy reconstructions of climate when examining the Hockey Stick controversy and found that there was a clear network of co-authorship between the Hockey Stick authors and almost all others working in the field, including those most likely to have been selected as a referee by an editor.
There was neither true independent verification of results nor peer review, and the possibilities for (at the very least) what we call "groupthink" were great. When Professor David Deming reported receiving an e-mail some years earlier from a senior climate scientist stating that there was a need to do something about the inconvenient truth presented by a Medieval Warm Period warmer than the present, the need for scepticism is obvious.
Scepticism can guard against such results, but unfortunately leading scientific journals seem to have lost their sceptical zeal and become, at least on occasions, boosters for good causes. Let me give you two examples from what many regard as the best journals of all: Nature and Science.
A 2004 paper in Nature using the species-area model to predict species distribution in response to modelled climate change (in turn based upon emissions scenarios) concluded its abstract with a call to action: "These estimates show the importance of rapid implementation of technologies to decrease greenhouse gas emissions and strategies for carbon sequestration." The paper itself presented neither reason nor evidence for such conclusions.
The problem is confined to neither climate science nor modelling. Science, for example, not only published the fraudulent research on cloning of Dr Woo Suk Hwang, but rushed it into print after short review so that it appeared in an electronic version, accompanied by a press release that ensured media coverage, on the eve of a key vote in the US Congress to overturn an administrative order of the Bush Administration prohibiting the use of federal funds for cloning research. Not only did it seem such research was more promising than was the case at that time, but South Korea was seemingly passing the US by.
Not only have leading science journals yielded to the temptation of the need for "relevance", but the ramparts of the prevailing paradigms are now defended using information technology to marshal the troops. "Swarming" is not confined to partying adolescents in yellow sunglasses, enjoying their 15 megabytes of fame, but is to be seen whenever ideas emerge to challenge the consensus. The white cells of the immune system of the dominant paradigm are despatched electronically, dealing with the infectious ideas with all means at their disposal, including (but by no means limited to) typically anonymous posters to internet discussions.
One of the means commonly employed is the use of the term "denier", a rhetorically powerful signifier quite deliberately first used (as far as I can tell) by a couple of defenders of the faith reviewing Bjørn Lomborg's The Sceptical Environmentalist for Nature. It was used quite deliberately by Jeff Harvey and Stuart Pimm to liken Lomborg to a holocaust denier for daring to question the highly questionable estimates of the number of species extinctions that supposedly occur every year.
The computer-based estimates of species extinction range all the way from a few tens of thousands to 50-100,000 (if you can believe Greenpeace). The actual documented number accepted by the International Union for the Conservation of Nature is around 800 over the 500 years for which we have records.
While I'm prepared to accept we have missed more than a few, and I'm a passionate advocate for the conservation of charismatic megafauna (such as tigers and orangutans), I think the use of the term "denier" tells us more about the person using it than about the target. I think the use of it amounts to an example of Godwins's Law of Internet Discussions, which holds that eventually someone will liken someone else to Hitler, at which point rational debate is over. (Implicitly, the person using it loses).
Unfortunately, the use of the term is rife in debates over climate change, where those on one side seem finally to have cottoned on to the point that scepticism in science is actually a good thing, and it was even used last year by the now minister responsible.
If it has served any purpose, this use of illiberal name calling serves to remind us of what is needed to ensure that noble cause corruption does not afflict the science informing public policy.
Those of us who see value in both social democracy and liberal democracy -- who are committed to humanist ideals but are open to evidence-based reasoning rather than ideology in determining how we are to advance them -- must acknowledge that it is from liberal views of the celebration of different points of view, and the battle of contending ideas, that good science derives.
The philosopher of science, Paul Feyerabend, warned that scientists might engage in all manner of devices -- from the rhetorical to the reprehensible -- to have their points of view prevail. It seems to me that the only protection against any kind of corruption in science is to celebrate the liberalism inherent in Karl Popper's philosophy of science, regardless of whether we share his political liberalism -- though separating the two might be difficult in practice. Feyerabend's prescription was a kind of anarchism and a rejection of any kind of marriage between science and the state.
As I said at the beginning of this lecture, the history of science is replete with error and fraud. In science, the best kind of quality assurance is to celebrate sceptical dissent and to reject any attempt to tell us that we should bow to a consensus, that "the science is settled" on principle -- not just even, but especially when it supports our preferences. Because as Carl Sagan once put it, "Where we have strong emotions, we're liable to fool ourselves."
No comments:
Post a Comment