Re: making snort go fast

[Matt Jonkman <jonkman () jonkmans com>]

David Williams wrote:
> Hello List,

Hello David.

> I'm trying to get Snort to go very fast.  Has anybody evaluated any of
> these solutions below.  I know these vendors are claiming multi-gig
> Snort, but I'm skeptical of vendor claims (obviously).
>
> - Endace's Ninja appliance (they claim 10G, but the webcast seemed to
> contradict this claim by stating just under 2G)

Glad to hear someone was listening to that webcast, I was the speaker.
:) Unfortunately I apparently didn't make my point clear enough. The
point of the benchmark I did wasn't to get a max throughput number. In
fact it was specifically NOT to get a max throughput number.

Let me get on my IDS benchmarking soap-box for a moment:
There really is NO way to make a fair and true representation of how an
IDS device of ANY sort (accelerated or not) will perform in Your
environment with YOUR traffic spread and YOUR ruleset. Just NO way,
because those three major variables have significant impact on
throughput. And when I say significant I mean that any single one of
them can bring a nice fast box down to a max throughput of 50meg/sec all
by itself. Down off my soapbox.

So that point made, in all of the benchmarking I've done my goal has
always been specifically NOT to find some mythical max throughput
number. It might look cool, but it means absolutel NOTHING to any
environment beside the test rig. Really really absolutely nothing. With
one rule and stripped down preprocessors and all homogenous traffic you
can make an IDS appliance push traffic as fast as it's bus can move
packets. But that means nothing, abso-frikkin-lutely nothing.

The goal of this endace benchmark I did was to take a baseline with a
REALLY high load ruleset (to make sure we tested all aspects of snort
and it's preprocs, etc) and a VERY hostile traffic corpus (a university
network capture with lots of attacks and 'liberal' users). The baseline
in that test was about 100meg/sec before packets were lost without any
acceleration in the mix.

Then we added the acceleration advantages of the particular platform,
but kept the same traffic and the same really high load ruleset and same
snort config, and found the new max. The percentage gain is the number
we wanted, which in that case was almost 16-fold increase in throughput.
What that number actually was wasn't important. How much better it was
is what was important.

So that tells us that in an environment/ruleset/snortconfig combination
on a 3ghz processor that does about 100meg/sec will see a 16-fold
increase if they introduce this platform's acceleration features. Up to
1.54Gbps+ in this case.

So that gives me as an IDS shopper the information I need to compare.
For instance, if I have a sensor with my ruleset, my traffic load and my
snort config running about 250meg/sec comfortably on a 3ghz processor, I
can likely expect to comfortably reach 4Gbps (250meg * 16) as a rough
number to compare.

So bottom line: My philosophy is that max throughput numbers are
useless, because the variables in the test environment are just too wide
to mean anything to anyone. You have to get a percent gain over a
baseline to make a comparison. There are still flaws in this model, but
it's the best thing I've ever had to use and it's served well enough to
date.

BTW: The endace ninjaboxes I don't believe advertise they can DO 10gbps,
but that they have 10gpbs fiber slots (2x per box I think). If you run a
VERY slim ruleset and a very tuned snort on some really easy traffic, it
might get there. Just as any other device could with the right
environment. Remember to clarify with any salesguy you talk to that he
either does or does not mean to imply that a 10gig port means it can
process packets at 10gbps. I think many sales guys let that implication
go hoping no one notices.

Matt


> - Netronome Systems Open Appliance (claiming 6-8G)
>
> - Bivio Networks B7000 (claiming 10G)
>
> Anybody else I'm missing from the list of vendors claiming to make
> Snort go fast?
>
> thanks,
>
> Dave
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users

-- 
--------------------------------------------
Matthew Jonkman
Emerging Threats
Phone 765-429-0398
Fax 312-264-0205
http://www.emergingthreats.net
--------------------------------------------

PGP: http://www.jonkmans.com/mattjonkman.asc



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users