FOLLOW-UP TO THE RECENT NASA CLIMBDOWN

An excerpt below from a long post in which Steve McIntyre looks at the defensive reaction from warmists to the recent NASA adjustments of the U.S. temperature record. In the excerpt below he notes with remarkable politeness the problems inherent in using U.S. weather station records that are sited in artificially warm locations. He notes that adjustments have allegedly been made for such locational problems but is politely appalled by the secrecy surrounding the adjustments. Science that does not make available enough detail of its research to enable replication of that research by others is just not science. He notes that at least one adjustment was UPWARDS rather than downwards, as one would expect. He also notes that, corrupted though the US data may be, it is undoubtedley less corrupted than much of the ROW (rest of the world) data -- suggesting that the recently improved U.S. record may be a better model of world effects than the corrupt figures at present in use. It is at any event now clear that we do not have GLOBAL warming. Warming that leaves out the USA is not global. At most we have ROW warming. GISS is NASA's Goddard Institute for Space Studies, led by the frantically Warmist James Hansen.

Can GISS Adjustments "Fix" Bad Data?

Now my original interest in GISS adjustments did not arise abstractly, but in the context of surface station quality. Climatological stations are supposed to meet a variety of quality standards, including the relatively undemanding requirement of being 100 feet (30 meters) from paved surfaces. Anthony Watts and volunteers of surfacestations.org have documented one defective site after another, including a weather station in a parking lot at the University of Arizona where MBH coauthor Malcolm Hughes is employed,

These revelations resulted in a variety of aggressive counter-attacks in the climate blogosphere, many of which argued that, while these individual sites may be contaminated, the "expert" software at GISS and NOAA could fix these problems

"Fixing" bad data with software is by no means an easy thing to do (as witness Mann's unreported modification of principal components methodology on tree ring networks.) The GISS adjustment schemes (despite protestations from Schmidt that they are "clearly outlined") are not at all easy to replicate using the existing opaque descriptions. For example, there is nothing in the methodological description that hints at the change in data provenance before and after 2000 that caused the Hansen error. Because many sites are affected by climate change, a general urban heat island effect and local microsite changes, adjustment for heat island effects and local microsite changes raises some complicated statistical questions, that are nowhere discussed in the underlying references (Hansen et al 1999, 2001). In particular, the adjustment methods are not techniques that can be looked up in statistical literature, where their properties and biases might be discerned. They are rather ad hoc and local techniques that may or may not be equal to the task of "fixing" the bad data.

Making readers run the gauntlet of trying to guess the precise data sets and precise methodologies obviously makes it very difficult to achieve any assessment of the statistical properties. In order to test the GISS adjustments, I requested that GISS provide me with details on their adjustment code. They refused. Nevertheless, there are enough different versions of U.S. station data (USHCN raw, USHCN time-of-observation adjusted, USHCN adjusted, GHCN raw, GHCN adjusted) that one can compare GISS raw and GISS adjusted data to other versions to get some idea of what they did.

In the course of reviewing quality problems at various surface sites, among other things, I compared these different versions of station data, including a comparison of the Tucson weather station shown above to the Grand Canyon weather station, which is presumably less affected by urban problems. This comparison demonstrated a very odd pattern discussed here. The adjustments show that the trend in the problematic Tucson site was reduced in the course of the adjustments, but they also showed that the Grand Canyon data was also adjusted, so that, instead of the 1930s being warmer than the present as in the raw data, the 2000s were warmer than the 1930s, with a sharp increase in the 2000s.

Now some portion of the post-2000 jump in adjusted Grand Canyon values shown here is due to Hansen's Y2K error, but it only accounts for a 0.5 deg C jump after 2000 and does not explain why Grand Canyon values should have been adjusted so much. In this case, the adjustments are primarily at the USHCN stage. The USHCN station history adjustments appear particularly troublesome to me, not just here but at other sites (e.g. Orland CA). They end up making material changes to sites identified as "good" sites and my impression is that the USHCN adjustment procedures may be adjusting some of the very "best" sites (in terms of appearance and reported history) to better fit histories from sites that are clearly non-compliant with WMO standards (e.g. Marysville, Tucson).

There are some real and interesting statistical issues with the USHCN station history adjustment procedure and it is ridiculous that the source code for these adjustments (and the subsequent GISS adjustments - see bottom panel) is not available

Closing the circle: my original interest in GISS adjustment procedures was not an abstract interest, but a specific interest in whether GISS adjustment procedures were equal to the challenge of "fixing" bad data. If one views the above assessment as a type of limited software audit (limited by lack of access to source code and operating manuals), one can say firmly that the GISS software had not only failed to pick up and correct fictitious steps of up to 1 deg C, but that GISS actually introduced this error in the course of their programming.

According to any reasonable audit standards, one would conclude that the GISS software had failed this particular test. While GISS can (and has) patched the particular error that I reported to them, their patching hardly proves the merit of the GISS (and USHCN) adjustment procedures. These need to be carefully examined. This was a crying need prior to the identification of the Hansen error and would have been a crying need even without the Hansen error.

The U.S. and the Rest of the World

Schmidt observed that the U.S. accounts for only 2% of the world's land surface and that the correction of this error in the U.S. has "minimal impact on the world data", which he illustrated by comparing the U.S. index to the global index. I've re-plotted this from original data on a common scale. Even without the recent changes, the U.S. history contrasts with the global history: the U.S. history has a rather minimal trend if any since the 1930s, while the ROW has a very pronounced trend since the 1930s

These differences are attributed to "regional" differences and it is quite possible that this is a complete explanation. However, this conclusion is complicated by a number of important methodological differences between the U.S. and the ROW. In the U.S., despite the criticisms being rendered at surfacestations.org, there are many rural stations that have been in existence over a relatively long period of time; while one may cavil at how NOAA and/or GISS have carried out adjustments, they have collected metadata for many stations and made a concerted effort to adjust for such metadata.

On the other hand, many of the stations in China, Indonesia, Brazil and elsewhere are in urban areas (such as Shanghai or Beijing). In some of the major indexes (CRU,NOAA), there appears to be no attempt whatever to adjust for urbanization. GISS does report an effort to adjust for urbanization in some cases, but their ability to do so depends on the existence of nearby rural stations, which are not always available. Thus, ithere is a real concern that the need for urban adjustment is most severe in the very areas where adjustments are either not made or not accurately made.

(For more postings from me, see TONGUE-TIED, EDUCATION WATCH INTERNATIONAL, GREENIE WATCH, POLITICAL CORRECTNESS WATCH, FOOD & HEALTH SKEPTIC, GUN WATCH, SOCIALIZED MEDICINE, AUSTRALIAN POLITICS, DISSECTING LEFTISM, IMMIGRATION WATCH INTERNATIONAL and EYE ON BRITAIN. My Home Pages are here or here or here. Email me (John Ray) here.)