Re: climate gate and programming bugs

[Dan Kaminsky <dan () doxpara com>]

On Wed, Dec 9, 2009 at 1:17 PM, Larry Seltzer <larry () larryseltzer com>wrote:

> > > since these "scientists" do not release their code. We are supposed
> to believe the priests who say Earth is at the center of the universe,
> but we are not allowed to see either their data or method they used to
> arrive at that conclusion.
>
> > > This isn't the production code, although it's related. CRU has
> promised to release both the code and the raw data. At that point, us
> coders can start the process of replicating the results, and looking for
> statistically significant errors.
>
> I agree this is the key point. I also think it's fair to state that
> without the leaked e-mails and documents they would not have agreed to
> release their data and code.
>
> I'll go one step further: No science is "settled" if nobody has even had
> the opportunity to replicate the work.
Sure, sounds great in theory.  In practice, do you have any idea how little
code and data is open?

Maybe you don't.

Here's the reality.  Academia is publish or perish. Publish is defined as
"getting papers into conferences".  It is not defined as "releasing the raw
data behind your paper" or "releasing even rough code that barely compiles"
or especially "releasing production code that other people can use on their
own data".  If you spend your time doing the latter, you might get cited a
bit more (since people use your stuff) but if it costs you a few papers,
you're going to perish.

That's even before the whole IP thing gets involved.

The reality is that for a whole bunch of reasons, a lot of stuff just isn't
available.  If you want it, if you want to reimplement it, you get
documentation in the form of a paper showing how to achieve what is
claimed.  Is the paper enough?  Sometimes it is, yeah.  But always?  Even
often?  No, not at all.

Of course, there's a revolution going on, because the *cost* of releasing
code and data is plummeting.  Expectations may change.  But I see it just as
likely that IP will take over, going so far as to delay and degrade the
papers themselves.

_______________________________________________
Fun and Misc security discussion for OT posts.
https://linuxbox.org/cgi-bin/mailman/listinfo/funsec
Note: funsec is a public and open mailing list.