Re: Snort + PF_RING + DAQ

[beenph <beenph () gmail com>]

On Tue, Sep 4, 2012 at 4:15 PM, Peter Bates <peter.bates () ucl ac uk> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> Hello all
>
> On 04/09/2012 19:06, Livio Ricciulli wrote:
> > o Do you have rules with long lists of IPs like
> > [ip1,ip2,ip3,ip4...] etc? These types of rules are horribly slow
> > because snort matches them linearly. If so, try disabling them and
> > see if things improve (if so, let me know we a plugin for that).
> > You can also configure snort with --enable-perfprofiling to see  if
> > there are bad rules that are taking too much time.
>
> I've compiled with --enable-sourcefire so I will try PPM and see if I
> have some really bad rules - they are mostly standard VRT and ET rules
> though and not things with big blacklists (dshield/ciarmy/rbn/spamhaus
> rules).
>
> > o Does top show the processes are mostly mostly pegged in the
> > 80%-100% kind of evenly?
>
> Yes - I have 32 threads/cores so top doesn't actually allow '1'
> because it just says my terminal is not long enough.
>
> > o Could it be contention caused by barnyard? Can you try without
> > and see if that helps?
>
> I'll try this but Barnyard seems incredibly light - all it does after
> all is pick up the unified2 files that Snort is placing in each directory.
>
> > o A big buffer can always help. When you load the pf_ring kernel
> > module give it at least 65k and place your interfaces in
> > transparent mode 1 as in: transparent_mode=1 min_num_slots=65536
> > (or even more than 65k if your kernel can handle it). You might
> > need to also increase the kernel memory with vmalloc=256M as a boot
> > parameter.
>
> I'll try increasing from 16384. It's never been that clear to me how
> the rings/buffers relate to available RAM. I do have quite a bit of
> RAM to play with.
>
> > o On some of our processors we got very good performance
> > improvements by compiling snort with "-march=native
> > -fomit-frame-pointer -O3"
>
> Okay - I can try this.
>
> > o What does cat /proc/interrupts show? Do you map the eth* IRQs to
> >  different CPUs or does CPU 0 do all the interrupts?
>
> The Intel ixgbe(10Gb) driver comes with a script called
> set_irq_affinity which I use to set the card IRQs to the CPUs - in
> /proc/interrupts it looks like a descending staircase pattern.


> The most recent PF_RING DAQ has a parameter to specifically bind
> Snort/DAQ instances to CPU ids so I'm using that in a similar loop to
> the one used to start Snort on the Metaflows site.
>
> > o Then there is the snort.conf.. I will let other people chime on
> > that..
>
> I'd be interested to hear anyone's suggestions for snort.conf tuning -
> the settings I've got at the moment are either stock/VRT 2.9.3.1 with
> some settings increased as per that supplied with redborder
> (redborder.net IDS/IPS).

IIRC you should have as many snort thread as network QUEUE your card
have, and you should balance
your IRQ on CPU and not CORE, thus if you have 16 dual core cpu, then you
chould bind 2 cpu (4 core) to each snort process.

I do not know how got your network card driver but mabey you would
like to compile it from source.

Ref: http://www.intel.com/support/network/adapter/pro100/sb/cs-032530.htm

Also you have alot of tunning depending on how your setup so you can
tune your driver to your needs.

-elz


> - --
> Peter Bates
> Senior Computer Security Officer    Phone: +44(0)2076792049
> Information Services Division       Internal Ext: 32049
> University College London
> London WC1E 6BT
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (Darwin)
> Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJQRmF/AAoJELhVoVpEMS6R/CYH/RoC34RfyLQRf/SYtTCKjdCf
> 3c+FGTUds7dsfA0CmKfd3CgjLZUq+uoA2kv3K93zAA8tQtVWnlQAnd+rIehCh4EW
> h2cVVDBKqPlt+2xZI0icHTvseyiBxStZIEEjrmbJjfntATLKOykfPCi/rknhrm6J
> qijnwhQJff9162+mZLaUetBIsGkrxzW2+QxZel8Ym3kclstmmrXUHf2xGAKJzsv5
> ZzP5VQFZpPJuuaTYisRhpc5qHbjgGiCbMtMKVlITxa7mf7Fis+o2OFwkBk6B2RwS
> LKZC0+/S3XtJ3e2RE5rbrE0VawD6aaxDu9TkgWzkDbPoyH7jVIU4eURNhB6pCTs=
> =ZI8P
> -----END PGP SIGNATURE-----
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
>
> Please visit http://blog.snort.org to stay current on all the latest Snort news!

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!