Re: Intrusion Detection Evaluation Datasets

[Sam Gorton <sam.gorton () gmail com>]

 On Thu, Mar 12, 2009 at 08:40:04AM -0700, Zow Terry Brugger wrote:
> I see a lot of people saying (correctly) that advanced (non-signature
> based) NIDS can't be researched until we have good evaluation
> datasets, and I see a lot of people ignoring them and doing it anyway.
> Is anyone (else) actually working on fixing the data problem?

There's been some progress, but it's unfortunately not public.  The
DHS PREDICT project (www.predict.org) includes various captured data
sets, including about 200 gig of artificial data sets we generated to
support a research project. PREDICT data's only available to
researchers based in the US who meet the program requirements.

There's no good answer right now to the problem of having a good
shared dataset, but I think that 'bad data' is a worse answer than 'no
data'. When the data does have problems, if the problems are clearly
labeled then hopefully researchers won't try to build systems around
artifacts.

--
Sam Gorton                   |   Skaion Corporation
sgorton () skaion com   |   www.skaion.com