Snort mailing list archives

Previous By Date Next
Previous By Thread Next

Re: Potential memory leak/settings for memory conservation in 2.9.2.3/2.9.3_rc1?

From: Jesse Bowling <jessebowling () gmail com>
Date: Thu, 5 Jul 2012 16:18:26 -0400
Sure thing...Here's my borkness, as best as I recall...I did not save my
borked copy so I reconstructed this from memory:

preprocessor stream5_global: memcap 1073741824, track_tcp yes, \
#   track_udp yes, \
   track_udp no, \
   track_icmp no, \
   max_tcp 262144, \
#   max_udp 131072
#   max_active_responses 2, \
#   min_response_seconds 5
preprocessor stream5_tcp: policy windows, detect_anomalies, require_3whs
180, \
   overlap_limit 10, small_segments 3 bytes 150, timeout 180, \
    ports client 21 22 23 25 42 53 79 109 110 111 113 119 135 136 137 139
143 \
        161 445 513 514 587 593 691 1433 1521 2100 3306 6070 6665 6666 6667
6668 6669 \
        7000 8181 32770 32771 32772 32773 32774 32775 32776 32777 32778
32779, \
    ports both 80 81 311 443 465 563 591 593 636 901 989 992 993 994 995
1220 1414 1830 2301 2381 2809 3128 3702 4343 5250 7907 7001 7145 7510 7802
7777 7779 \
        7801 7900 7901 7902 7903 7904 7905 7906 7908 7909 7910 7911 7912
7913 7914 7915 7916 \
        7917 7918 7919 7920 8000 8008 8014 8028 8080 8088 8118 8123 8180
8243 8280 8800 8888 8899 9080 9090 9091 9443 9999 11371 55555
#preprocessor stream5_udp: timeout 180

One should keep in mind that I may have totally mis-read what the problem
was; perhaps it's not about UDP tracking, but when I changed my settings to
the following, my problem went away:

preprocessor stream5_global: memcap 1073741824, track_tcp yes, \
   track_udp yes, \
   track_icmp no, \
   max_tcp 262144, \
   max_udp 131072
#   max_active_responses 2, \
#   min_response_seconds 5
preprocessor stream5_tcp: policy windows, detect_anomalies, require_3whs
180, \
   overlap_limit 10, small_segments 3 bytes 150, timeout 180, \
    ports client 21 22 23 25 42 53 79 109 110 111 113 119 135 136 137 139
143 \
        161 445 513 514 587 593 691 1433 1521 2100 3306 6070 6665 6666 6667
6668 6669 \
        7000 8181 32770 32771 32772 32773 32774 32775 32776 32777 32778
32779, \
    ports both 80 81 311 443 465 563 591 593 636 901 989 992 993 994 995
1220 1414 1830 2301 2381 2809 3128 3702 4343 5250 7907 7001 7145 7510 7802
7777 7779 \
        7801 7900 7901 7902 7903 7904 7905 7906 7908 7909 7910 7911 7912
7913 7914 7915 7916 \
        7917 7918 7919 7920 8000 8008 8014 8028 8080 8088 8118 8123 8180
8243 8280 8800 8888 8899 9080 9090 9091 9443 9999 11371 55555
preprocessor stream5_udp: timeout 180

Cheers,

Jesse

On Thu, Jul 5, 2012 at 2:43 PM, Russ Combs <rcombs () sourcefire com> wrote:


Jesse, thanks for following up.  Can you send borked settings so we can
try to prevent such outcomes?

Russ

On Thu, Jul 5, 2012 at 1:28 PM, Jesse Bowling <jessebowling () gmail com>wrote:


Hello everyone,

Not sure if this list is active, but wanted to note that the issue I
mentioned earlier went away after I tweaked the stream5 settings for the
snort instances. I had removed some lines from the stream5 processing
configuration in an attempt to not track UDP; instead I caused UDP
'sessions' to be track without limit.

Needless to say, this caused some performance issues. :)

Sorry for the false alarm,

Jesse


On Tue, Jul 3, 2012 at 5:55 PM, Jesse Bowling <jessebowling () gmail com>wrote:


Hello,

While running snort 2.9.2.3 on modest hardware with PF_RING I've found
that after 1 - 3 hours the snort processes have used enough memory to cause
swapping, which in turn leads to iowait, which leads to additional system
time, which ends in a death spiral with snort and PF_RING dropping and
failing to analyze almost all traffic on a link averaging 200-400 MB/s of
traffic. This appears to also be the case with 2.9.3_rc1.

Some particulars are included below, but before the wall of text I
wanted to ask:

Is there a known memory leak in these version?

Are there snort.conf options I can/should tweak to limit the amount of
memory that snort uses on this limited resource machine?

What tools or techniques can I use to help profile the performance issue
and isolate it's source? I'm fairly certain the issue lies within snort,
but I'd like to have something more definitive than top/vmstat/sar output.

How can I download previous versions of snort? I've built this
monitoring stack before and did not observe issues of this nature then; I'd
like to fall back to an older version and confirm that it functions
properly.

Thanks in advance,

Jesse

Tech details:

Linux sensor-test 2.6.32-279.el6.x86_64 #1 SMP Wed Jun 13 18:24:36 EDT
2012 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 6.3 (Santiago)

PF_RING Version     : 5.2.1 ($Revision: 5041$)
Ring slots          : 8192
Slot version        : 13
Capture TX          : No [RX only]
IP Defragment       : No
Socket Mode         : Standard
Transparent mode    : No (mode 2)
Total rings         : 2
Total plugins       : 0

# snort --version

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.3_rc GRE (Build 35)
   ''''    By Martin Roesch & The Snort Team:
http://www.snort.org/snort/snort-team
           Copyright (C) 1998-2012 Sourcefire, Inc., et al.
           Using libpcap version 1.1.1
           Using PCRE version: 7.8 2008-09-05
           Using ZLIB version: 1.2.3

# snort --version

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.2.3 GRE (Build 205)
   ''''    By Martin Roesch & The Snort Team:
http://www.snort.org/snort/snort-team
           Copyright (C) 1998-2012 Sourcefire, Inc., et al.
           Using libpcap version 1.1.1
           Using PCRE version: 7.8 2008-09-05
           Using ZLIB version: 1.2.3

$ ./configure --with-libpcap-includes=/usr/
local/include --with-libpcap-libraries=/usr/local/lib
--with-dnet-includes=/usr/local/include
--with-dnet-libraries=/usr/local/lib --disable-ipv6
--disable-active-response --disable-react

DAQ:
It was created by daq configure 0.6.2, which was
generated by GNU Autoconf 2.67.  Invocation command line was

  $ ./configure --with-libpcap-includes=/usr/local/include
--with-libpcap-libraries=/usr/local/lib


--
Jesse Bowling






--
Jesse Bowling




------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel

Please visit http://blog.snort.org for the latest news about Snort!







-- 
Jesse Bowling

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel

Please visit http://blog.snort.org for the latest news about Snort!
Previous By Date Next
Previous By Thread Next

Current thread: