Re: SNORT Testing

[Martin Roesch <roesch () sourcefire com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Byron,

This may sound a bit snippy but it's not pointed at you, I'm just  
frustrated with the tools that are out there. :)

Stick and Snot do *not* test Snort, they haven't tested Snort in any  
meaningful way for years, and they only "tested" Snort in their  
original form for a few months in 2001 while I made things more  
stateful.  If you really want to test Snort for performance you  
should probably start thinking about investing a few hundred $k in  
some gear from Spirent or maybe Ixia for load generation and then get  
metasploit for attack generation.  A properly configured Snort on a  
fast enough platform will take gigabit switches and high end test  
equipment to generate enough traffic to simulate anything that will  
tax it.

Without the load generation gear all you can do is functional testing  
of Snort and for that you should probably be looking at metasploit/ 
fragrouter/scapy/etc for that sort of thing.

I don't know if FPG is capable of doing anything with rules that use  
flowbits or relative offsets from previous detections, much less  
regex rules.  This includes the vast majority of rules that are  
developed for Snort these days.  Mucus is in the same boat, it was  
built for Snort version ~1.8.3-6, it will be unsuitable for testing  
modern versions of Snort if the latest release (from 2003) is any  
indication.

Stick/snot/sneeze/fpg/mucus are not suitable ways of testing Snort's  
"throughput", let's all try to remember that from this point on,  
we've been saying it for years.  If you want to get a really accurate  
measurement of how Snort performs, you should be putting it into an  
operational environment where it's going to be deployed and tune it  
suitably for that environment and then see what the numbers look  
like.  That's the absolute best way, doing repeatable network-based  
testing is the next best way and after that you've got a variety of  
non-repeatable or irrelevant testing setups that won't show you  
anything meaningful because they're not repeatable nor are they well  
scoped.

What you want to achieve is repeatable functional testing of the  
engine components at high bandwidth utilization/packet per second  
rates.  The repeatable high-bandwidth generation costs lots of money,  
the functional testing tools are largely available for free, although  
there are a few good commercial tools out there too.

     -Marty


On Feb 27, 2006, at 5:54 PM, Byron Sonne wrote:

> The tools that come to mind for me are 'stick' and 'snot':
> http://archives.neohapsis.com/archives/fulldisclosure/ 
> 2004-09/0096.html
>
> ---------------------------------------------------------------------- 
> --
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks  
> from CORE IMPACT.
> Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus- 
> ids_040708 to learn more.
> ---------------------------------------------------------------------- 
> --

- --
Martin Roesch - Founder/CTO, Sourcefire Inc. - +1-410-290-1616
Sourcefire - Security for the Real World - http://www.sourcefire.com
Snort: Open Source Network IDS - http://www.snort.org




-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)

iD8DBQFEA98Aqj0FAQQ3KOARAslaAJ45YA1xQQOTcEGOUAgp8vu0RYunvgCeMKPZ
NcPLTGTz/Ytm1Gruxa808j8=
=AaZT
-----END PGP SIGNATURE-----

------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it 
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
------------------------------------------------------------------------