Re: Updated IP Blacklisting patch (version 2)

[Eoin Miller <eoin.miller () trojanedbinaries com>]

Martin Roesch wrote:
> On Tue, Jul 7, 2009 at 1:02 PM, Eoin
> Miller<eoin.miller () trojanedbinaries com> wrote:> Marty,
>   
> > After the update to the code, there was an 15% or so up-tick in system
> > processor utilization which is leading to exceptionally mild increase in
> > packet loss. Not really a surprise since when SnortEventqAdd is called,
> > it now does two lookups to find the number of the list and then the
> > number of the list again to refrence the name:
> >
> > SnortEventqAdd(GENERATOR_SPP_IPLIST, (int)pn->data, 1, 0, 0,
> > list_names[(int)pn->data], 0);
> >
> > So to try and streamline this a bit more, I added a new int inside of
> > IpListEval:
> >
> > void IpListEval(Packet *p, void *conext)
> > {
> > int foo;
> >
> >
> > Then inside of the if(blacklist_detect) we grab (int)pn->data once and
> > then save it as foo and just use foo when calling SnortEventqAdd:
> >
> >        if(blacklist_detect)
> >        {
> >            if(!noalerts)
> >                foo = (int)pn->data;
> >                SnortEventqAdd(GENERATOR_SPP_IPLIST, foo, 1, 0, 0,
> >                               list_names[foo], 0);
> >
> >
> > This drops the system processor utilization back down by about 10% or so
> > and completely stops the packet loss we were experiencing. Some links to
> > gnuplot graphs based off the snort.stats file are included below to
> > support this are below:
> >
> > http://trojanedbinaries.com/security/snort/cpu.png
> > http://trojanedbinaries.com/security/snort/dropped.png
> >     
>
> Wow, 15%!  That's a heck of a lot of overhead for a single added
> pointer dereference.  Is that 15% greater than what Snort was using
> before or 15% of total system CPU?  I took a little closer look at the
> function and made a couple changes.  Not sure of the performance
> impact but evaluating the whitelist and bailing on a whitelist detect
> before evaluating the blacklist should result in some savings.  Let me
> post the updated function:
>
>
> But goto's are Bad so we'd never do that... :)
>
> Marty
>
>
>   
Yeaup, that was 15% more total utilization for that core. Snort was
using ~35% of a core to monitor ~450Mbit/s of traffic. After adding the
second pointer dereference it was using ~50% of a core to monitor the
same amount of traffic. FYI, this test snort instance has no rules
loaded and is using Phil Wood's MMAP'd libpcap with a 1GigaByte buffer
of system RAM.

If you look at the cpu.png file
(http://trojanedbinaries.com/security/snort/cpu.png) you can see the
spike in the green line (system%) and the dip in the blue line (idle%) @
16:00. That was when snort was relaunched with the double pointer
derefrence in the call to the SnortEventqAdd function:

SnortEventqAdd(GENERATOR_SPP_IPLIST, (int)pn->data, 1, 0, 0,
list_names[(int)pn->data], 0);

But if you notice the dip in the green line and rise in the blue line
from 16:40-16:50, that was when I was running recompiled with the single
derefrence:

foo = (int)pn->data;
SnortEventqAdd(GENERATOR_SPP_IPLIST, foo, 1, 0, 0, list_names[foo], 0);

Tried your new first function you posted and the results appear the
same. Good deal less processor utilization and no more packet loss and
your new function makes more sense for those using the whitelisting
functionality. Tried to use the fancy free way with the goto's, but gcc
got all whiny about something.

--
Eoin Miller


------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time, 
vendors submitting new applications to BlackBerry App World(TM) will have 
the opportunity to enter the BlackBerry Developer Challenge. See full prize 
details at: http://p.sf.net/sfu/blackberry
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users