RE: 2.1.3rc1 Performance RESULTS

[Gary_Portnoy () itginc com]

I've trimmed some people from the To list, didn't want to keep flooding 
their inboxes.

Chad, 

Thanks for the tuning tips.  I've been playing around with different 
settings since morning, and had increased sq_max_size to first to 50 then 
to 100.  This has prevented the norcvbuf and nocanput counters from 
"netstat -k" from going up any higher.  They are currently holding at 
their values (around 330 MILLION for each.  I hate to think how many 
events I missed).  However, Snort statistics still keep on reporting 
dropped packets, pretty much at the same rate as it was before.  I 
increased the tcp_max_buf and the tcp_recv_hiwat values to 4 times their 
present size.  They are at 4194304 and 262144 respectively.  I don't want 
to do too much too fast, but I haven't seen any changes yet.

What bothers me is that now that nocanput and norcvbuf have stopped going 
up, why is Snort still dropping packets???  I wonder if now it is 
something within Snort, since it seems that my machine is finally keeping 
up with the network load. 

Can someone from sourcefire/snort team comment on how the performance 
statistics (both perfmon processor and after receiving a USR1 signal) are 
created?  How reliable are they?  Do they report just what they receive 
from libpcap, or would they report as "dropped" packets that they received 
from libpcap, but couldn't process for whatever reason.

I just sent 'kill -USR1 snort.pid' and here are the latest stats:
Snort analyzed 20223045 out of 21439997 packets, dropping 1216952(5.676%) 
packets.
That's after running for about 3 hours.

-Gary-
-------------------------------------------
Gary Portnoy
Information Security Analyst
ITG, Inc.




"Kreimendahl, Chad J" <Chad.Kreimendahl () umb com>
05/21/2004 02:38 PM

 
        To:     <Gary_Portnoy () itginc com>, "Dirk Geschke" <Dirk () geschke-online de>, 
"Darren Webb" <spyder007 () charter net>
        cc:     <snort-users () lists sourceforge net>, "Daniel J. Roelker" 
<droelker () sourcefire com>
        Subject:        RE: [Snort-users] 2.1.3rc1 Performance  RESULTS


Ok, try these following things:  I've included some notes below

-----------------------------------------
This is from /etc/system (I've included the comments from above it)

*Increasing Synchronized Queues to Improve Network Performance
*
*  To increase the size of STREAMS synchronized queues, thereby
*  increasing network performance, add the sq_max_size variable to
*  the /etc/system file.
*
*       set sq_max_size=n
*
*  Set the sq_max_size variable to n, where n is the maximum number
*  of messages that are allowed for each IP queue.
*  values should be incremented in small steps ( 10 ) and never set
higher than 100.

set sq_max_size=100

---------------------------------------------
The following we run in a startup script:

/usr/sbin/ndd -set /dev/tcp tcp_max_buf 8194304
/usr/sbin/ndd -set /dev/tcp tcp_recv_hiwat 8194304 


The first one lets more packets sit in each queue.  Setting this high
would make things like web connections to that server seem a little
slower, but will greatly improve high bandwidth sniffing.  The second
changes the high water mark for the buffers in whatever protocol.
(tcp_<whatever> can be changed to udp_<whatever> and works identically).
Of note, xmit_hiwat (alternate to recv_hiwait) is unimportant on links
such as spans, but might be considered if you're actually using that
interface to route traffic.

There may be some other changes that we've done.  However, those 3
settings posted gave us the greatest effect on our end ability to sniff
traffic at high rates.

These settings shouldn't be necessary on a system like a Sun
V210/240/250.. (fire systems)... etc when installed with Solaris 9.  The
drivers for the built-in bge interface(s) are excellent, and the kernel
handles changing between polling/interrupts in brilliant fashion. 


-----Original Message-----
From: Gary_Portnoy () itginc com [mailto:Gary_Portnoy () itginc com] 
Sent: Friday, May 21, 2004 9:00 AM
To: Kreimendahl, Chad J; Dirk Geschke; Darren Webb
Cc: snort-users () lists sourceforge net; Daniel J. Roelker
Subject: [Snort-users] 2.1.3rc1 Performance RESULTS

So last night I ran the following test:

Captured 1 million packets off the wire, verified the capture with 
capinfo.
Connected my snort sensor (Sun Ultra-2, 1024MB RAM, 2CPU, Quad Ethernet 
card) with a crossover cable to a Sun V120.
> From the V120, injected the packet capture to a snort process running on

the sensor.  The snort config file has no rules, but both the stream4
and 
frag2 preprocessors were on.  I alternated between running the two 
different snort processes.The rate of injection was: 591379.9 bytes/sec 
4.51 megabits/sec 2401 packets/sec as reported by tcpreplay

I did this 3 times with each Snort 2.1.3rc1 with libpcap 0.7.2 library
and 
Snort 2.1.3rc2 with libpcap 0.8.3 library. 

Snort 2.1.3rc1 with libpcap 0.7.2 library:
Attempt 1: Snort analyzed 970415 out of 970415 packets, dropping
0(0.000%) 
packets
Attempt 2: Snort analyzed 975566 out of 975566 packets, dropping
0(0.000%) 
packets
Attempt 3: Snort analyzed 978249 out of 978249 packets, dropping
0(0.000%) 
packets
Total: 2924230 captured, out of 3,000,000 sent. 

Snort 2.1.3rc2 with libpcap 0.8.3 library:
Attempt 1: Snort analyzed 999706 out of 1000000 packets, dropping 
294(0.029%) packets
Attempt 2: Snort analyzed 1000000 out of 1000000 packets, dropping 
0(0.000%) packets
Attempt 3:Snort analyzed 1000000 out of 1000000 packets, dropping 
0(0.000%) packets
Total: 2999706 captured out of 3,000,000.

So, not only is libpcap 0.8.3 more efficient, it also accurately reports

packets dropped (see below), while Snort with libpcap 0.7.2 had no
concept 
of dropped packets, at least on Solaris.

I did a few more tests with libpcap 0.8.3:
First turning on the rules i normally use, about 2/3 of the current 
rule-base:
Snort analyzed 985212 out of 1000000 packets, dropping 14788(1.479%) 
packets
Snort analyzed 978054 out of 1000000 packets, dropping 21946(2.195%) 
packets
Snort analyzed 987043 out of 999996 packets, dropping 12953(1.295%) 
packets 
*** This last one is interesting because it seems to have missed 4
packets 
somewhere, so maybe it's not AS honest as i thought

Then with the rules from above, turning up the rate to 7Mbps:
Snort analyzed 996595 out of 1000000 packets, dropping 3405(0.340%) 
packets
Snort analyzed 988972 out of 1000000 packets, dropping 11028(1.103%) 
packets

Not sure why it reports droping less packets at 7Mbps than at 4.5.

That's all the tests I ran thus far.  If anyone has anything else to 
suggest, I am willing to try it if I have some free time.

In the meantime, does anyone know any way to tune Solaris to make it
more 
efficient at packet captures?

-------------------------------------------
Gary Portnoy






-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-+-
This message is for the named person's use only. This communication is
for 
informational purposes only and has been obtained from sources believed
to 
be reliable, but it is not necessarily complete and its accuracy cannot
be 
guaranteed. It is not intended as an offer or solicitation for the
purchase
or sale of any financial instrument or as an official confirmation of
any
transaction. Moreover, this material should not be construed to contain
any
recommendation regarding, or opinion concerning, any security. It may
contain confidential, proprietary or legally privileged information. No
confidentiality or privilege is waived or lost by any mistransmission.
If
you receive this message in error, please immediately delete it and all
copies of it from your system, destroy any hard copies of it and notify
the
sender. You must not, directly or indirectly, use, disclose, distribute,

print, or copy any part of this message if you are not the intended 
recipient.  Any views expressed in this message are those of the
individual
sender, except where the message states otherwise and the sender is 
authorized to state them to be the views of any such entity.

ITG Inc. reserves the right to monitor and archive all electronic 
communications through its network. 

ITG Inc. Member NASD, SIPC
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-+-










-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
This message is for the named person's use only. This communication is for 
informational purposes only and has been obtained from sources believed to 
be reliable, but it is not necessarily complete and its accuracy cannot be 
guaranteed. It is not intended as an offer or solicitation for the purchase
or sale of any financial instrument or as an official confirmation of any
transaction. Moreover, this material should not be construed to contain any
recommendation regarding, or opinion concerning, any security. It may
contain confidential, proprietary or legally privileged information. No
confidentiality or privilege is waived or lost by any mistransmission. If
you receive this message in error, please immediately delete it and all
copies of it from your system, destroy any hard copies of it and notify the
sender. You must not, directly or indirectly, use, disclose, distribute, 
print, or copy any part of this message if you are not the intended 
recipient.  Any views expressed in this message are those of the individual
sender, except where the message states otherwise and the sender is 
authorized to state them to be the views of any such entity.

ITG Inc. reserves the right to monitor and archive all electronic 
communications through its network. 

ITG Inc. Member NASD, SIPC
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-



-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g. 
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users