Re: Intrusion Detection Evaluation Datasets

[Paul Palmer <paul_palmer () us ibm com>]

> > > again.  So the main nuisances on the wire keep changing, and any 
dataset
> > > is necessarily going to get stale very quickly.
> >
> > Very true, and so if any dataset is needed this has to be kept in 
mind.
> An anonymous reviewer on one of my papers made the following
> observation, which I've taken to heart: The IDS community doesn't need
> another static dataset, we need a way to periodically create good
> datasets. Much in the way that the KDD community puts out a dataset
> for their annual KDD Cup challenge, we should be able to generate a
> dataset -- say every year -- representative of both the threats and
> background traffic at that time. I rather fancy the idea of letting
> all the NIDS researchers have at it and presenting an award for the
> best performance at RAID.
>
> Cheers,
> Terry

Isn't this essentially one of the things that NSS does as part of their 
product reviews?

Granted, getting your product reviewed is not free, but I think you would 
find that it is probably cheaper to have NSS perform the evaluation than 
investing in the infrastructure to do it yourself.