Re: Intrusion Detection Evaluation Datasets

["\"Zow\" Terry Brugger" <zow () acm org>]

> If you extend your familiarity with the NIDS/NIPS industry further, I think
> you will ultimately find that Stefano is correct. I think that you will find
> that the majority of the top NIDS/NIPS products use "signature" engines that
> are not based upon SNORT technology.

I understand most IDS vendors do not actually use the Snort code
(apology forthcoming). In fact, from what I can tell, apart from
SourceFire and some vendors who include Snort with hardware appliances
designed to run it, I can't find any commercial vendors who use the
Snort code as the basis of their product (and are still around).
(Anyone please feel free to educate me on this point.)

> I think you will also ultimately find
> that while SNORT is very good at what it was designed to do, it is not a
> universal solution. Trade-offs were made. The product has strengths and
> weaknesses. SNORT is not the sine qua non of NIDS. Mercifully, Marty has
> left some of the market for the rest of us :)

I do apologize -- my previous message did rather imply that all
vendors used or simply rewrote the Snort code and that all the tweaks
were made therein. This is, obviously, not the case, and such a
statement is quite disrespectful to the hundreds of developers at
numerous companies who have spent countless hours trying to build the
best IDS in the marketplace. The main areas I've actually seen vendors
differentiate themselves are:
- performance (please stand up if you can do 10GigE)
- strong vulnerability research teams (or deep pockets to buy exploits)
- enterprise-level management of configuration, signatures, and alerts
- basic SIM capabilities to filter out false positives

The core point I was trying to make is that I haven't seen any attacks
of interest that one signature based IDS could detect that another
couldn't. I say attacks of interest because I am aware of some DoS
attack detection available on some systems, which is only really
useful on an IPS, because I don't need my IDS to tell me that some
punk is using a botnet to hammer my systems (also, those DoS attack
detectors -- at least the good ones -- are not signature based). Now
if anyone would like to educate me about some signature-based
technologies that can detect attacks my Snort system can not, I'd be
eager to learn (as I'm sure many others would, as well). If you want
to hock your own product, please feel free to contact me off list.

Someone mentioned Bro specifically. I don't think Bro provides
anything new and interesting in the signature detection realm. What it
provides is an architecture which is much more amenable to building
more advanced detection capabilities. The real interesting things I've
seen come out of Bro only used Bro for basic data collection, which an
analyst was then able to find interesting patterns from. This goes
strongly to Staniford's point about Paxson diving into the live data.

Cheers,
Terry