Climategate and the cost of not being open
cmw.osdude 120000QT77 Visits (2535)
I'm not here to talk about the merits of what the scientists may or may not have done or what the hackers may or may not have done. I want to talk about how the proprietary information has created complications. You see, the researchers decided at some point that they could not make their information available because it was proprietary. Now that these accusations have come out they are in the difficult position of trying to prove a negative.
"You lied about your data! You've been manipulating this the whole time."
"Uhhhh.... No we haven't."
"Yes you did! You've destroyed or hidden everything that doesn't support your claims."
"No! We didn't. Look we'll show you some of it."
"OK. That looks OK. But what about the rest of it?"
"We can't show you that because it's secret."
"But that's where you're hiding everything."
"Uhhhhh... No we're not."
"Look, we talked it over and we've decided to show you all the data."
"Are you sure that's all of it?"
"You didn't hold anything back."
"Uhhhhhh... we're pretty sure."
"We think you're lying."
"No we're not."
"We don't believe you."
You see, the problem is that when someone makes accusations about something that is secret you can't prove that it's incorrect without revealing the secrets. Usually this is done with some sort of trusted third party and there are judges and lawyers involved. However, when the issue is one of conspiracy anyway then the assumption is just that the third parties really couldn't be trusted after all. It's complicated and unfortunate.
When information is open those sorts of things are harder to claim and easier to disprove. Let's say try this scenario:
"Firefox had a secret program that is sending all of my keystrokes to Homeland Security."
"That's not too hard to disprove. The code for Firefox is open. I can get a team of experts to look through all of that code and determine whether or not they can see any possible way that could be true. So far... nothing!"
"Ah! But the source code lies. You didn't compile that yourself. They're too smart. They only do it for the people who download the binaries. It's in the binaries."
"OK. I'm going to compile the source code that I have. I'll get information from the Firefox team about the parameters and tools that they used and I'll follow the same procedure. The end results should be the exact same binaries. Look! They match! There's nothing hidden."
"Drat! You've foiled my theory. Now I will go and pick on food additives."
In business there are sound reasons for keeping some information a secret. There are also advantages to openness... especially in areas of science and technology. Things such as Creative Commons and even traditional copyright (if applied correctly) allow establishment of ownership without obfuscating the data. I know that I'm oversimplifying these issues to a great degree... but it just got me to thinking. If the scientists at East Anglia had decided to be more open with their data in the first place would they be in this weird place right now?
As authors of the future are we better off keeping our work hidden, or being more open? I know what I think.