Re: Snort 2.9, RHEL 5 and afpacket DAQ [~Solved?]

[Russ Combs <rcombs () sourcefire com>]

If you think this patch is good have Ron review it and we can add it to
2901.

On Wed, Oct 20, 2010 at 6:06 PM, Michael Altizer <maltizer () sourcefire com>wrote:

>  On 10/20/2010 04:16 PM, Michael Altizer wrote:
>
> On 10/20/2010 03:14 PM, Michael Altizer wrote:
>
> On 10/20/2010 02:59 PM, Rich Graves wrote:
>
> On Wed, Oct 20, 2010 at 1:12 PM, Jeff Kell wrote:
>
> > I had rebuilt snort 2.8.6 with libpcap 1.1.1 and  had some worse
> > performance than before, but then there was a discussion on one of the snort
> > lists regarding sids 4676 and 4677 in the oracle-rules being a pcre "hog".
> >
> > Disabling those two sids dropped my average CPU over half...
>
> Wow. Mine dropped over 2/3.
>
> sid 4676 is limited to POSTs, so if you have a requirement to detect
> ancient oracle attacks, keep that one and drop just 4677.
>
> The problem of the maximum 49MB buffer on RHEL5 64-bit remains (does not
> affect Ubuntu 64-bit or RHEL5 32-bit; I'd expect it to effect CentOS and
> other rebuilds as well), but since I'm no longer regularly filling the
> buffer, my 2.9.0 installation is now stable enough that I can start looking
> at the new rule options, and hope the buffer issue gets addressed in 2.9.1.
>
>  I've replicated the issue on a 64-bit CentOS 5.5 VM.  It's going to take
> some investigation from the kernel side of af_packet to figure out the issue
> since it appears to be limited to 64-bit CentOS/RHEL as you indicated.
> Unfortunately, they really don't make building a custom kernel with their
> source easy, but I'm getting there...
>
> -Michael
>
> The problem is that we're running into the old kmalloc limit of 128k on the
> 2.6.18 kernel supplied with RHEL5/CentOS5.  The difference between 49mb and
> 50mb is the difference between 16218 blocks and 16549 blocks, which the
> kernel attempts to kmalloc a pointer array for to keep track of the
> allocated pages.  On a 64-bit system:
> 16218 * sizeof(char *) = 129744 (< 128k)
> 16549 * sizeof(char *) = 132392 (> 128k)
>
> The Linux kernel started defaulting to supporting kmallocs of significantly
> larger sizes (2mb) in 2.6.22.  See the AFPacket section of the DAQ README if
> you're curious as to the math involved with determining the number of blocks
> (pages) requested.
>
> There are a few avenues that can be explored:
> 1. Change the AFPacket DAQ module to attempt to use initially use
> multi-page blocks and scale back the block size on subsequent allocation
> failures.
> 2. Upgrade your kernel to something a bit more recent (2.6.22 came out
> almost 3 and a half years ago).
> 3. Change the af_packet.c on your RHEL kernel to use vmalloc instead of
> kmalloc for the page vector and recompile, but that will have an unknown
> impact on performance.
>
> I'll start looking into option 1 (probably the best option overall) when I
> get some time.
>
> -Michael
>
> I've attached an updated version of my previous patch which incorporates
> item 1.  I've tested it on CentOS 5.5 (where the problem was reproduced and
> debugged) and have not had any issues.
>
> -Michael
>
>
> ------------------------------------------------------------------------------
> Nokia and AT&T present the 2010 Calling All Innovators-North America
> contest
> Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in
> marketing
> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
> http://p.sf.net/sfu/nokia-dev2dev
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store 
http://p.sf.net/sfu/nokia-dev2dev

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users