First off, I have to express a little disappointment, because I planned to write this last night after reading some posts from yesterday and now it looks more like it’s a response to a recent guest post. I always planned on going with this title (sans the P.S.), but now it sounds a bit Orwellian in context. No matter.

OK, obviously it would be a huge travesty and very suspicious if raw unadjusted data was permanently destroy, but it was not. I will repeat, there is no raw data is gone for good. Period. End of statement.

So what did happen? CRU took the raw data from various primary sources, aggregated it and then made adjustments. It is some of the aggregation that they threw out when they moved a few decades ago. This means that the original data still exists at the primary sources and can be reaggregated. In fact, CRU is busy doing that just now.

Where did I get this information? From one of the largest thorns in the AGW community’s side, Roger Pielke Jr., who seems completely satisfied with their explanation.

Skeptics like Dr. Pielke claim that isn’t the entirety of the problem. Apparently most papers in the field rely on the adjusted data by CRU, and CRU hasn’t released the entirety of how they made the adjustments (they claim 95% of data sets have been available, I’m not qualified to say what that means). The skeptics assume they’ll disagree with some of the adjustments and want to see exactly what raw data CRU used and in what way they used it. This is of course an entirely reasonable request. It is also something that CRU now says they will do.

Honestly, this is great for the climate science field on several levels. First of all, it is far past time that an easy to use cohesive database of every single step was made publicly available. That should have been in place around the beginning of the decade. Now some people may claim that it should always have been available, but the fact is that’s not how science works, nor how it has ever worked. Contrary to a lot of popular opinion, science is a cutthroat business and data is very much IP in order to get grants and get a leg up. This is partially because a) there is a hierarchy in the industry, just like everything else, and places with more resources can easily scoop you before you get proper credit b) science is very conservative and skeptical by nature, and any discrepancy at all is often used to shut down new explanations, first policed within the field and secondly by other fields c) it takes a ton of time to handle all the requests and put data in an easy to access way, something that is already at a premium and d) a lot of interpretation is built on standardized assumptions within the field, different fields may have different assumptions and it takes a long time to develop a communication paradigm for a different field*.

So what happens, pretty much since time immemorial (a new meme highlights Newton’s correspondence) is that scientists first are very protective of data and share only with collaborators, then as they get a bit of a foothold and other people get similar conclusions then they are more open within the field and finally when that field has a cohesive explanation then it tries to communicate more broadly. Well until the start of this decade there were still some frays on the cohesive explanation, but those have been nearly uniformly agreed upon, so I think they should have been more proactive in describing each part in a way where scientists in other fields could use their own understanding and look for critiques.

Hopefully, this moment will force climatologists to develop a standardized way of disseminating their data and models to the public, which will have the dual benefit of getting more technically oriented people involved and improve the science (there obviously will be some errors, especially on the programming level).

I also am seeing some changes in rhetoric that suggest that ClimateGate is waking up advocates on how to think about messaging. For instance, while “deniers” and “skeptics” used to be used interchangeably as epithets, I’m now reading on various sites that there is a very targeted movement to use “denier” to label people/groups that the author feels has no interest in the actual science and is just trying to create noise for commercial gain (like the tobacco companies did) versus “skeptics” for people that genuinely do have an interest and just don’t believe the consensus. See Climate Progress for an example.

This is also a welcome development because it will sharpen the rhetorical knives between how to communicate with people that are “ignorant” and those that are “deceptive” and of course those that just have different conclusions even after careful analysis. I put “ignorant” and “deceptive” in quotes because they are obviously not objective categories — unless internal memos are found of course — but are still useful because it will allow for people that are on the fence to be able to see which views climatologists and their proponents categorize as which. This will make it so that people that are skeptical feel less under attack and (in my biased opinion) they will see that the professional deniers are full of it by purposeful obfuscation.

Within the field, AGW was very controversial even until the late 90s, at which point many former skeptics were convinced based on new information and modelling. I feel that AGW is truly correct, and a more open approach will do the same for intellectual society writ large (there will always be a strong political component and a large minority of people with their heads in the sand for ideological reasons, but that’s true generally). Hopefully this tiff will be the start of that progression, especially when outside influences actually make the science better.

Of course the more interesting time will be when it is largely accepted and people are just arguing ferociously about how to respond to it on a socio-political level.

* I’ve noticed that most skeptics are chemists, geologists and basic engineers. Except for a minority of scientists in those fields, none of them operate on the level of dynamical systems, feedback loops, etc. As such, their arguments why it can’t be real are so ungrounded in the basic principles that it takes forever just to get them on page one. By contrast, biologists (especially systems biologists), nonlinear control theory engineers and many physicists have paradigms that rely on dynamical theory and I have yet to meet one that is skeptical of AGW on a fundamental level.

Update: I’d be remiss to point out that the CRU data set isn’t the only comprehensive collection in the world, and that it actually shows far less warming than the NASA/NOAA ones due to how it handles the Arctic.