Re: high packet loss - low throughput

[rmkml <rmkml () yahoo fr>]

ids or ips/inline mode please?

can you run top with "top -d 1" and press key "1" (for enabling all show 
cpu) and wait 5s and send full output please ? (otherwise top summarize 
info)

do you have compiled snort ? what option ? how to run snort cmd line 
please ? what snort version ? can you send your snort.conf please ? snort 
verbose output ? snort verbose output statistics ? like dump stat during 
snort alive with kill -USR1 every 5mn...
what is your search-mode on your snort.conf please?
how many snort events you have during your tests please ?

do you have compiled pfring ? options ? cmd line ? pfring stats ? 
(pfcount, /proc/net/pf_ring/stats)

Prevouisly requested, can you remove all snort rules ? what cpu ? what loss ?
try disable rules + mod so ? cpu ? loss ?
try disabling preproc ? cpu ? loss ?

yes please try disable netsniff-ng for testing

try enlarge buffer like:
http://www.mail-archive.com/ntop-misc () listgateway unipi it/msg01289.html
http://mikelococo.com/files/2011/2011_01_25-snort_performance.pdf

YM previously requested:
Which libpcap and and tcpdump are installed, i.e.: from the pfring tarball or standalones, or just the package from the 
SO distro?
I read in some post that in some cases its recommended not to manually 
bind Snort to processors, instead let the kernel figure it out. Have you tried that?
One last thing, what features are enabled on the NICc, tso, gro, etc.?

Regards
@Rmkml


On Fri, 19 Jul 2013, waldo kitty wrote:

> On 7/19/2013 15:51, Michal Purzynski wrote:
> > On 7/19/13 6:37 PM, waldo kitty wrote:
> > > On 7/19/2013 05:16, Michal Purzynski wrote:
> > > > So, anyone got some ideas how to debug and improve the situation? Or
> > > > should I just assume that snort isn't capable of handling a per process
> > > > 30Mbit - I can see a 5% packet loss now.
> > > are you running a 64bit OS on those boxes or a 32bit one? which OS? you said
> > > (below) part of a security onion so i'm going to guess linux... now 64 or 32bit?
> > >
> > > assuming *nix, what does top show?
> > >
> > >      top -bn1 | head
> > top -bn1 | head
> >
> > top - 19:50:58 up 10:15,  1 user,  load average: 5.74, 5.30, 4.59
>
> you definitely have something keeping the processors busy... let's run that top
> command again with a larger head...
>
> top -bn1 | head 20
>
> > Tasks: 321 total,   3 running, 318 sleeping,   0 stopped,   0 zombie
> > Cpu(s): 15.5%us,  0.8%sy,  0.0%ni, 81.7%id,  0.1%wa,  0.0%hi, 1.9%si,
> > 0.0%st
>
> well, it isn't disk bound (wa) which is good... i wouldn't expect it to be
> either... all of these seems to be ok i think... just not sure about 15.5% being
> spent in user time but i guess that depends on the apps running and as what
> user... looking at the below, i see that snort is running as user sguil so that
> may be part of the deal on that one...
>
> > Mem:  65939336k total, 65673944k used,   265392k free,    33508k buffers
> > Swap: 33969596k total,        0k used, 33969596k free, 46105348k cached
>
> no swap used which is good...
>
> >     PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+ COMMAND
> > 33155 sguil     20   0  379m 354m 337m R   39  0.5 238:13.59 netsniff-ng
> > 34035 sguil     20   0  849m 727m  11m R   33  1.1 102:38.41 snort
> > 35091 sguil     20   0  851m 731m  11m S   25  1.1 111:49.86 snort
>
> we'll see more of these with the above command... i note that netsniff-ng has
> twice as much time consumed as the shown snort processes... what happens if you
> disable netsniff-ng?
>
> > 64 bit of course. It's Ubuntu 12.04.2, everything updated, etc.
>
> ok...
>
> > I've noticed an interesting statistics BTW:
> > - there are some processes doing +- 60Mbit/sec with a packet loss over 6%
> > - there are some doing 90-100 with 0% packet loss (or at least below 1%,
> > which is my goal)
> >
> > I don't understand it, what might be a reason?
>
> not sure yet but something is binding things down... here's a bassackwards
> thought... i asked if you had tried running more instances to see if each
> additional instance would take some of the load off the others and allow all of
> them to process more... you said you had gone from 6 to 8... my first thought
> was to try more... if you were running 6 then double it and see what happens...
> now i'm wondering what happens if you drop the number of instances back so that
> each processes more but there are fewer to keep the system busy... what happens
> if you run only one instance? then two instances? and so on...
>
> also, what are your NICs? do they have hardware offloading capabilities for some
> functions so that snort and/or your OS doesn't have to deal with them? are you
> using those capabilities?
k

------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!