Re: high packet loss - low throughput

[Michal Purzynski <michal () rsbac org>]

On 7/19/13 6:37 PM, waldo kitty wrote:
> On 7/19/2013 05:16, Michal Purzynski wrote:
> > So, anyone got some ideas how to debug and improve the situation? Or
> > should I just assume that snort isn't capable of handling a per process
> > 30Mbit - I can see a 5% packet loss now.
> are you running a 64bit OS on those boxes or a 32bit one? which OS? you said
> (below) part of a security onion so i'm going to guess linux... now 64 or 32bit?
>
> assuming *nix, what does top show?
>
>     top -bn1 | head
top -bn1 | head

top - 19:50:58 up 10:15,  1 user,  load average: 5.74, 5.30, 4.59
Tasks: 321 total,   3 running, 318 sleeping,   0 stopped,   0 zombie
Cpu(s): 15.5%us,  0.8%sy,  0.0%ni, 81.7%id,  0.1%wa,  0.0%hi, 1.9%si,  
0.0%st
Mem:  65939336k total, 65673944k used,   265392k free,    33508k buffers
Swap: 33969596k total,        0k used, 33969596k free, 46105348k cached

   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+ COMMAND
33155 sguil     20   0  379m 354m 337m R   39  0.5 238:13.59 netsniff-ng
34035 sguil     20   0  849m 727m  11m R   33  1.1 102:38.41 snort
35091 sguil     20   0  851m 731m  11m S   25  1.1 111:49.86 snort
64 bit of course. It's Ubuntu 12.04.2, everything updated, etc.

I've noticed an interesting statistics BTW:
- there are some processes doing +- 60Mbit/sec with a packet loss over 6%
- there are some doing 90-100 with 0% packet loss (or at least below 1%, 
which is my goal)

I don't understand it, what might be a reason?
> > On 7/18/13 11:07 AM, Michal Purzynski wrote:
> > > On 7/18/13 3:39 AM, waldo kitty wrote:
> > > > On 7/17/2013 17:25, Michal Purzynski wrote:
> > > > > On 7/17/13 11:01 PM, waldo kitty wrote:
> > > > > > On 7/17/2013 16:04, Michal Purzynski wrote:
> > > > > > > Hello,
> > > > > > >
> > > > > > > I can see a strange results on a local snort installation. Either I
> > > > > > > don't understand something or the statistics aren't precise.
> > > > > > > Please help
> > > > > > > me understand.
> > > > > > >
> > > > > > > It's an (expanding) two hosts snort setup with 2 x E5-2620 0 @
> > > > > > > 2.00GHz /
> > > > > > > 64GB RAM each.
> > > > > > > Intel x520 card.
> > > > > > > Traffic is around 1Gbit to each host.
> > > > > > > Around 3500 VRT only rules enabled.
> > > > > > > 8 snort instances load balanced by the pf_ring.
> > > > > > what else is this machine doing besides just snorting the traffic?
> > > > > netsniff-ng, barnyard, snort and that's it. Part of a Security Onion,
> > > > > but with most things (like Bro, argus, prads, etc) disabled.
> > > > > > > The traffic loss is very high - up to 9% per instance (as reported by
> > > > > > > Sguil which in turn read the snort logs and debug files). A single
> > > > > > > instance gets from 90 - 150Mbits of traffic and from 10 - 20k pps. To
> > > > > > > make it worse, the loss is not dependent on the traffic and/or pps at
> > > > > > > all. Actualy, sometimes I get a 5% of loss on 50Mbits to a single
> > > > > > > instance.
> > > > > > what happens if you increase the number of snort instances which
> > > > > > would thereby
> > > > > > reduce the load on each of the instances?
> > > > > I did it increasing from 6 to 8. And it won't help, really - if snort
> > > > > cannot keep up with 50Mbit / instance stream...
> > > > i'm not sure that it is snort, specifically... there is something
> > > > causing the
> > > > data to be flushed or lost before it has a chance to be processed...
> > > > there are
> > > > others running snort on pipes as large or larger...
> > > >
> > > > perhaps you are using protocol aware stream flushing and it needs
> > > > tweaking?
> > > Yes, it's enabled with the same settings. Reading about it and I don't
> > > really want to disable it.
> > > > ###############################################
> > > > # Configure protocol aware flushing
> > > > # For more information see README.stream5
> > > > ###############################################
> > > > config paf_max: 16000
> > > >
> > > >
> > > > it may also be related to the timeout values in the stream5 settings??
> > > No idea, that's why asking here :) Everything is default.


------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!