RE: Snort getting overloaded by http traffic:

[Matt Kettler <mkettler () evi-inc com>]

I agree almost completely about the underlying OS and you raise many good 
points, except about the IP stack part. Snort does not use the IP stack for 
packet capture, it's a pcap layer system so IP stack performance is 
completely irrelevant to the performance of snort. It's a very common 
misconception that higher performance IP stacks help snort out, but they 
really do not unless you are doing some form of remote logging to a 
database (in which case it speeds the logging process). I know it's picking 
minor details, but it is a significant difference.

In fact, there's good reason to believe that some systems with very high 
performance IP stacks, such as FreeBSD, have high latency in getting data 
to snort due to the interface they use for pcap (BPF in the case of 
FreeBSD). note: that latency really should only matter to people doing 
flexresp, or some form of back-end log parsing for realtime blocking, but 
does provide an example of pcap/ipstack performance disconnect.

In general, to expand a bit on what Keith mentioned, there are several 
things that matter to snort's packet drop rate (this probably isn't a 
complete listing, but is a start). Depending on what kind of setup you 
have, some are likely to be a non-issue, and others will be your biggest 
bottlenecks.

        1) Ethernet hardware and device driver efficiency
                bad PCI busmastering implementations incur additional data 
copies for buffer alignment, slowing all forms of network activity down.
        2) Underlying OS libpcap interface type (BPF, etc)
        3) Underlying OS resource management, Disk caching, scheduling 
latency, etc.
        4) Load from other processes on the same system as snort
        5) CPU speed, memory size, memory bandwidth, and disk bandwidth.
        6) Snort rule configuration
        7) Snort logging configuration.
        8) Your particular traffic profile.

I also agree with Keith that since Ashley was pretty vague about their 
system setup, it's hard to say what bottleneck is causing the drops. For 
all we know Ashely could be using a 486 with an ISA NIC to try to monitor a 
saturated T3 line (ouch!)  :)

Ashely's suggestion of cutting down the rule list will certainly help drive 
the memory and CPU requirements of snort down a bit, but that will likely 
only help if that's the bottleneck. Also making sure that HTTP_SERVERS is 
tightly defined to only cover real webservers inside the network will help 
save a lot of CPU time if that's not already been done. Getting rid of 
rules that go off a lot that you know your server is immune to will help 
save CPU and Disk time (ie: cmd.exe monitoring of an apache/linux webserver 
really is of limited value, since the failures will be logged by apache).

Keith's basic benchmarking suggestion is an excellent one, monitoring the 
performance of your real setup as you tweak things is likely to be the only 
real measure of how snort will perform for you. With a bit more information 
about what kind of setup you have, the list members can make some better 
"start here" type recommendations. Without much, I'd agree with Keith's 
first-stab suggestion of memory exhaustion as a possibility.





At 01:35 PM 6/25/2002 -0400, Keith wrote:
> The amount of traffic that Snort is able to inspect has less to do with 
> Snort and almost everything to do with the underlying operating system, IP 
> stack, and (most importantly) available resources.  If the operating 
> system is short of resources (specifically RAM), then packets are going to 
> be dropped by the kernel due to lack of buffer space and general 
> congestion.  As such, they will never be presented to Snort for inspection.
>
> That said, Snort can adversely (but indirectly) affect performance, as 
> something like full logging uses up processor time and valuable memory, 
> taking away from resources that could otherwise be used by pcap to capture 
> and process traffic moving up the stack.  So in this sense, Snort has an 
> effect, albeit a small one.
>
> Anyway, the best place to start would be to do some basic benchmarking, 
> and monitor your system's resources.  As I mentioned, insufficient RAM is 
> the likely suspect, so keep a particularly close eye on your memory stats.
>
> Also, when posting in the future, you can help us to help you by 
> always  providing information about the operating system, processor speed, 
> memory, Snort version, etc.  Many times, someone has been in the same 
> boat, and can offer up some pretty sound, specific advice.
>
> Cheers
>
> Keith



-------------------------------------------------------
This sf.net email is sponsored by: Jabber Inc.
Don't miss the IM event of the season | Special offer for OSDN members! 
JabConf 2002, Aug. 20-22, Keystone, CO http://www.jabberconf.com/osdn
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users