Re: Snort + PF_RING + DAQ

[Joel Esler <jesler () sourcefire com>]

I'm so glad to be seeing this type of conversation go on. 

Thank you all!

--
Joel Esler

On Sep 4, 2012, at 8:00 PM, livio Ricciulli <livio () metaflows com> wrote:

> On 09/04/2012 04:12 PM, Luca Deri wrote:
> > Hi Livio,
> > if you have a look at http://www.ntop.org/wp-content/uploads/2012/09/Snort_over_DNA_Silicom_30_07_2012_1.pdf you 
> > will see that the speed bump of DNA with respect to non-DNA PF_RING DAQ ranges from +20% to over +1400%. So I won't 
> > say "slightly better performance".
> >
> > Regards Luca
> Slight in the message below was referring to the difference between 24 
> snort processes (freely scheduled by Linux using NAPI) and 16 snort 
> processes using DNA (DNA wins).
> Thanks for the write-up!
> It kind of confirms our numbers if I understand them right (almost 
> linear speedup 14/16):
> The 1400% measures the improvement of single thread af_packet  (4 
> threads x 4 machines) to mutiple thread NAPI (16 threads x 4 machines)?
> And 20% compares multiple thread NAPI (16 threads x 4 machines) to 
> multiple-thread DNA (16 threads x 4 machines)?
> Right?
>
> With our workload (large EDU trace) the bottleneck seems to be more on 
> the CPU and less IO. When IO starts being more of a bottleneck, then DNA 
> clearly wins big (I would not call 20% slight).
>
> Livio.
> > On Sep 5, 2012, at 12:54 AM, livio Ricciulli<livio () metaflows com>  wrote:
> >
> > > > CPU Binding is something important, QUEUE wise if you bind a snort
> > > > process to the same network QUEUE
> > > > then you can clearly start to benchmark. If you spread the network
> > > > queue load on multiple CPU and do not bind process
> > > > to the same CPU then your adding context switching in the mix which i
> > > > think is bad at high throuput.
> > > In pfring lingo this is called DNA and does give slightly better performance
> > > which supports your claim:
> > > see https://www.metaflows.com/technology/10-gbps-pf_ring-2/
> > >
> > > We found, though, that with NAPI and letting the Linux scheduler loose
> > > on 24 threads
> > > works just as well but gives you much better flexibility (you can have
> > > multiple
> > > applications share the same interface for example which you cannot do
> > > with DNA).
> > >
> > > So, your theory is correct but it does not make a big enough difference,
> > > (on our appliances).
> > > And I doubt it would solve Peter's problem. But again, it is hard to
> > > generalize and I might be wrong..
> > >
> > > Livio.
> > >
> > >
> > > ------------------------------------------------------------------------------
> > > Live Security Virtual Conference
> > > Exclusive live event will cover all the ways today's security and
> > > threat landscape has changed and how IT managers can respond. Discussions
> > > will include endpoint security, mobile security and the latest in malware
> > > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > > _______________________________________________
> > > Snort-users mailing list
> > > Snort-users () lists sourceforge net
> > > Go to this URL to change user options or unsubscribe:
> > > https://lists.sourceforge.net/lists/listinfo/snort-users
> > > Snort-users list archive:
> > > http://www.geocrawler.com/redir-sf.php3?list=snort-users
> > >
> > > Please visit http://blog.snort.org to stay current on all the latest Snort news!
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
>
> Please visit http://blog.snort.org to stay current on all the latest Snort news!

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!