Snort mailing list archives

Previous By Date Next
Previous By Thread Next

Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3

From: Victor Roemer <vroemer () sourcefire com>
Date: Wed, 5 Jun 2013 13:08:53 -0400
Martinez, as Joel already mentioned, we'll want to see your Snort
configuration. Shutdown stats would also be useful, but perfmon data would
be better; if those can be provided.

You mentioned that OpenBSD configured the network sysctl parameters "on the
fly"; could you direct us to some documentation about this?

You also mentioned that Snort was listening on em3, however the startup
information in your email indicates that Snort is listening on em4, could
you elaborate on this setup?


Regarding Suricata, I personally do not have any experience in deploying or
configuring it. That said, are you using, relatively, the same
configurations? (e.g., any rules enabled, acquiring packets via libpcap,
etc..)

Also, why are "tcp.reassembly_gap" and "tcp.invalid_checksum" relevant?


On Wed, Jun 5, 2013 at 11:06 AM, Joel Esler <jesler () sourcefire com> wrote:


Can you post your snort.conf somewhere?


On May 31, 2013, at 2:51 AM, C. L. Martinez <carlopmart () gmail com> wrote:


On Thu, May 30, 2013 at 12:45 PM, C. L. Martinez <carlopmart () gmail com>

wrote:

Hi all,

According to the following stats:

May 30 11:46:22 nsm01 snort[30096]:


===============================================================================

May 30 11:46:22 nsm01 snort[30096]: Packet Performance Summary:
May 30 11:46:22 nsm01 snort[30096]:    max packet time       : 10000

usecs

May 30 11:46:22 nsm01 snort[30096]:    packet events         : 654
May 30 11:46:22 nsm01 snort[30096]:    avg pkt time          : 27.1384

usecs

May 30 11:46:22 nsm01 snort[30096]: Rule Performance Summary:
May 30 11:46:22 nsm01 snort[30096]:    max rule time         : 4096

usecs

May 30 11:46:22 nsm01 snort[30096]:    rule events           : 20
May 30 11:46:22 nsm01 snort[30096]:    avg rule time         : 1.046

usecs

May 30 11:46:22 nsm01 snort[30096]:


===============================================================================

May 30 11:46:22 nsm01 snort[30096]: Packet I/O Totals:
May 30 11:46:22 nsm01 snort[30096]:    Received:     69971576
May 30 11:46:22 nsm01 snort[30096]:    Analyzed:     22427618 ( 32.052%)
May 30 11:46:22 nsm01 snort[30096]:     Dropped:     41532168 ( 37.247%)
May 30 11:46:22 nsm01 snort[30096]:    Filtered:            0 (  0.000%)
May 30 11:46:22 nsm01 snort[30096]: Outstanding:     47543958 ( 67.948%)
May 30 11:46:22 nsm01 snort[30096]:    Injected:            0
May 30 11:46:22 nsm01 snort[30096]:


===============================================================================

May 30 11:46:22 nsm01 snort[30096]: Breakdown by protocol (includes
rebuilt packets):
May 30 11:46:22 nsm01 snort[30096]:         Eth:     22436767 (100.000%)
May 30 11:46:22 nsm01 snort[30096]:        VLAN:            0 (  0.000%)
May 30 11:46:22 nsm01 snort[30096]:         IP4:     22436767 (100.000%)
May 30 11:46:22 nsm01 snort[30096]:        Frag:           12 (  0.000%)
May 30 11:46:22 nsm01 snort[30096]:        ICMP:       110634 (  0.493%)
May 30 11:46:22 nsm01 snort[30096]:         UDP:       752816 (  3.355%)
May 30 11:46:22 nsm01 snort[30096]:         TCP:     19433478 ( 86.614%)

using snort under OpenBSD 5.3 doesn't returns good performance. Host
is a Intel(R) Xeon(R) CPU E5620 @ 2.40GHz, with 8 GiB RAM and four
e1000 interfaces.

In this sensor, I only use so_rules:

# dynamic library rules
# include $SO_RULE_PATH/bad-traffic.rules
# include $SO_RULE_PATH/chat.rules
include $SO_RULE_PATH/dos.rules
include $SO_RULE_PATH/exploit.rules
# include $SO_RULE_PATH/icmp.rules
# include $SO_RULE_PATH/imap.rules
include $SO_RULE_PATH/misc.rules
include $SO_RULE_PATH/multimedia.rules
include $SO_RULE_PATH/netbios.rules
# include $SO_RULE_PATH/nntp.rules
include $SO_RULE_PATH/p2p.rules
include $SO_RULE_PATH/smtp.rules
# include $SO_RULE_PATH/snmp.rules
include $SO_RULE_PATH/specific-threats.rules
include $SO_RULE_PATH/web-activex.rules
include $SO_RULE_PATH/web-client.rules
include $SO_RULE_PATH/web-iis.rules
include $SO_RULE_PATH/web-misc.rules

and monitored network is a 1GiB network.

Any ideas why??



More info:


top:
load averages:  0.69,  0.65,  0.53
31 processes: 30 idle, 1 on processor
CPU0 states:  2.8% user,  0.0% nice,  0.4% system, 20.4% interrupt,

76.4% idle

CPU1 states:  2.2% user,  0.0% nice,  0.8% system,  0.0% interrupt,

97.0% idle

CPU2 states:  3.0% user,  0.0% nice,  3.4% system,  0.0% interrupt,

93.6% idle

CPU3 states:  6.0% user,  0.0% nice,  5.0% system,  0.0% interrupt,

89.0% idle

Memory: Real: 587M/2947M act/tot Free: 5012M Cache: 2213M Swap: 0K/6142M

 PID USERNAME PRI NICE  SIZE   RES STATE     WAIT      TIME    CPU

COMMAND

14655 root       4    0  393M  183M sleep/1   bpf       8:44 14.26% snort
25669 root       4    0 1132K 1740K sleep/2   bpf       0:06  3.52%

daemonlogger


systat ifstat (snort process is listening in em3)

   3 users    Load 0.89 0.71 0.56                     Fri May 31

06:23:13 2013


IFACE            STATE  DESC
                    IPKTS   IBYTES    IERRS    OPKTS   OBYTES
OERRS    COLLS
em0              up
                        2      132        0        0      261
0        0
em1              up
                        0      126        0        0      131
0        0
em2              up
                    10348  3425952        0        0        0
0        0
em3              up
                    10346  3425044        0        0        0
0        0


systat mbufs


IFACE             LIVELOCKS  SIZE ALIVE   LWM   HWM   CWM
System                    0   256   185          56
                              2k   171         435
lo0
em0                            2k     6     4   256     6
em1                            2k     6     4   256     4
em2                            2k    66     4   256    66
em3                            2k    65     4   256    65


Stats with ALL so_rules disabled (5 min, more or less):

Rule application order:
activation->dynamic->pass->drop->sdrop->reject->alert->log
Verifying Preprocessor Configurations!
ICMP tracking disabled, no ICMP sessions allocated
IP tracking disabled, no IP sessions allocated
0 out of 1024 flowbits in use.

Packet Performance Monitor Config:
 ticks per usec  : 2417 ticks
 max packet time : 10000 usecs
 packet action   : fastpath-expensive-packets
 packet logging  : log
 debug-pkts      : disabled

Rule Performance Monitor Config:
 ticks per usec  : 2417 ticks
 max rule time   : 4096 usecs
 rule action     : suspend-expensive-rules
 rule threshold  : 5
 suspend timeout : 10 secs
 rule logging    : log
pcap DAQ configured to passive.
Acquiring network traffic from "em4".
Reload thread starting...
Reload thread started, thread 0xc100dbb8f00 (18056)
Decoding Ethernet

       --== Initialization Complete ==--

  ,,_     -*> Snort! <*-
 o"  )~   Version 2.9.4.6 GRE (Build 73)
  ''''    By Martin Roesch & The Snort Team:
http://www.snort.org/snort/snort-team
          Copyright (C) 1998-2013 Sourcefire, Inc., et al.
          Using libpcap version 1.3.0
          Using PCRE version: 8.31 2012-07-06
          Using ZLIB version: 1.2.3

          Rules Engine: SF_SNORT_DETECTION_ENGINE  Version 1.17  <Build

18>

          Rules Object: web-misc  Version 1.0  <Build 1>
          Rules Object: web-iis  Version 1.0  <Build 1>
          Rules Object: web-client  Version 1.0  <Build 1>
          Rules Object: web-activex  Version 1.0  <Build 1>
          Rules Object: specific-threats  Version 1.0  <Build 1>
          Rules Object: snmp  Version 1.0  <Build 1>
          Rules Object: smtp  Version 1.0  <Build 1>
          Rules Object: p2p  Version 1.0  <Build 1>
          Rules Object: nntp  Version 1.0  <Build 1>
          Rules Object: netbios  Version 1.0  <Build 1>
          Rules Object: multimedia  Version 1.0  <Build 1>
          Rules Object: misc  Version 1.0  <Build 1>
          Rules Object: imap  Version 1.0  <Build 1>
          Rules Object: icmp  Version 1.0  <Build 1>
          Rules Object: exploit  Version 1.0  <Build 1>
          Rules Object: dos  Version 1.0  <Build 1>
          Rules Object: chat  Version 1.0  <Build 1>
          Rules Object: bad-traffic  Version 1.0  <Build 1>
          Preprocessor Object: SF_DNP3  Version 1.1  <Build 1>
          Preprocessor Object: SF_MODBUS  Version 1.1  <Build 1>
          Preprocessor Object: SF_GTP  Version 1.1  <Build 1>
          Preprocessor Object: SF_REPUTATION  Version 1.1  <Build 1>
          Preprocessor Object: SF_SIP  Version 1.1  <Build 1>
          Preprocessor Object: SF_SDF  Version 1.1  <Build 1>
          Preprocessor Object: SF_DCERPC2  Version 1.0  <Build 3>
          Preprocessor Object: SF_SSLPP  Version 1.1  <Build 4>
          Preprocessor Object: SF_DNS  Version 1.1  <Build 4>
          Preprocessor Object: SF_SSH  Version 1.1  <Build 3>
          Preprocessor Object: SF_SMTP  Version 1.1  <Build 9>
          Preprocessor Object: SF_IMAP  Version 1.0  <Build 1>
          Preprocessor Object: SF_POP  Version 1.0  <Build 1>
          Preprocessor Object: SF_FTPTELNET  Version 1.2  <Build 13>
Commencing packet processing (pid=18056)
^C*** Caught Int-Signal


===============================================================================

Run time for packet processing was 421.51287 seconds
Snort processed 630885 packets.
Snort ran for 0 days 0 hours 7 minutes 1 seconds
  Pkts/min:        90126
  Pkts/sec:         1498


===============================================================================

Packet Performance Summary:
  max packet time       : 10000 usecs
  packet events         : 0
  avg pkt time          : 5.9247 usecs
Rule Performance Summary:
  max rule time         : 4096 usecs
  rule events           : 0


===============================================================================

Packet I/O Totals:
  Received:      1863847
  Analyzed:       630885 ( 33.849%)
   Dropped:       601452 ( 24.397%)
  Filtered:            0 (  0.000%)
Outstanding:      1232962 ( 66.151%)
  Injected:            0


===============================================================================


Not really good numbers ....


Stats with only misc.rules and multimedia.rules (5 min, more or less):

Rule application order:
activation->dynamic->pass->drop->sdrop->reject->alert->log
Verifying Preprocessor Configurations!
ICMP tracking disabled, no ICMP sessions allocated
IP tracking disabled, no IP sessions allocated
WARNING: flowbits key 'file.vqf' is checked but not ever set.
WARNING: flowbits key 'file.wmp_playlist' is checked but not ever set.
8 out of 1024 flowbits in use.

[ Port Based Pattern Matching Memory ]
+- [ Aho-Corasick Summary ] -------------------------------------
| Storage Format    : Full-Q
| Finite Automaton  : DFA
| Alphabet Size     : 256 Chars
| Sizeof State      : Variable (1,2,4 bytes)
| Instances         : 27
|     1 byte states : 26
|     2 byte states : 1
|     4 byte states : 0
| Characters        : 1562
| States            : 1446
| Transitions       : 16926
| State Density     : 4.6%
| Patterns          : 90
| Match States      : 88
| Memory (KB)       : 562.24
|   Pattern         : 10.08
|   Match Lists     : 19.52
|   DFA
|     1 byte states : 261.06
|     2 byte states : 225.67
|     4 byte states : 0.00
+----------------------------------------------------------------
[ Number of patterns truncated to 20 bytes: 4 ]

Packet Performance Monitor Config:
 ticks per usec  : 2422 ticks
 max packet time : 10000 usecs
 packet action   : fastpath-expensive-packets
 packet logging  : log
 debug-pkts      : disabled

Rule Performance Monitor Config:
 ticks per usec  : 2422 ticks
 max rule time   : 4096 usecs
 rule action     : suspend-expensive-rules
 rule threshold  : 5
 suspend timeout : 10 secs
 rule logging    : log
pcap DAQ configured to passive.
Acquiring network traffic from "em4".
Reload thread starting...
Reload thread started, thread 0x4aa997dc00 (32237)
Decoding Ethernet

       --== Initialization Complete ==--

  ,,_     -*> Snort! <*-
 o"  )~   Version 2.9.4.6 GRE (Build 73)
  ''''    By Martin Roesch & The Snort Team:
http://www.snort.org/snort/snort-team
          Copyright (C) 1998-2013 Sourcefire, Inc., et al.
          Using libpcap version 1.3.0
          Using PCRE version: 8.31 2012-07-06
          Using ZLIB version: 1.2.3

          Rules Engine: SF_SNORT_DETECTION_ENGINE  Version 1.17  <Build

18>

          Rules Object: web-misc  Version 1.0  <Build 1>
          Rules Object: web-iis  Version 1.0  <Build 1>
          Rules Object: web-client  Version 1.0  <Build 1>
          Rules Object: web-activex  Version 1.0  <Build 1>
          Rules Object: specific-threats  Version 1.0  <Build 1>
          Rules Object: snmp  Version 1.0  <Build 1>
          Rules Object: smtp  Version 1.0  <Build 1>
          Rules Object: p2p  Version 1.0  <Build 1>
          Rules Object: nntp  Version 1.0  <Build 1>
          Rules Object: netbios  Version 1.0  <Build 1>
          Rules Object: multimedia  Version 1.0  <Build 1>
          Rules Object: misc  Version 1.0  <Build 1>
          Rules Object: imap  Version 1.0  <Build 1>
          Rules Object: icmp  Version 1.0  <Build 1>
          Rules Object: exploit  Version 1.0  <Build 1>
          Rules Object: dos  Version 1.0  <Build 1>
          Rules Object: chat  Version 1.0  <Build 1>
          Rules Object: bad-traffic  Version 1.0  <Build 1>
          Preprocessor Object: SF_DNP3  Version 1.1  <Build 1>
          Preprocessor Object: SF_MODBUS  Version 1.1  <Build 1>
          Preprocessor Object: SF_GTP  Version 1.1  <Build 1>
          Preprocessor Object: SF_REPUTATION  Version 1.1  <Build 1>
          Preprocessor Object: SF_SIP  Version 1.1  <Build 1>
          Preprocessor Object: SF_SDF  Version 1.1  <Build 1>
          Preprocessor Object: SF_DCERPC2  Version 1.0  <Build 3>
          Preprocessor Object: SF_SSLPP  Version 1.1  <Build 4>
          Preprocessor Object: SF_DNS  Version 1.1  <Build 4>
          Preprocessor Object: SF_SSH  Version 1.1  <Build 3>
          Preprocessor Object: SF_SMTP  Version 1.1  <Build 9>
          Preprocessor Object: SF_IMAP  Version 1.0  <Build 1>
          Preprocessor Object: SF_POP  Version 1.0  <Build 1>
          Preprocessor Object: SF_FTPTELNET  Version 1.2  <Build 13>
Commencing packet processing (pid=32237)
^C*** Caught Int-Signal


===============================================================================

Run time for packet processing was 368.552024 seconds
Snort processed 643495 packets.
Snort ran for 0 days 0 hours 6 minutes 8 seconds
  Pkts/min:       107249
  Pkts/sec:         1748


===============================================================================

Packet Performance Summary:
  max packet time       : 10000 usecs
  packet events         : 0
  avg pkt time          : 8.95128 usecs
Rule Performance Summary:
  max rule time         : 4096 usecs
  rule events           : 0
  avg rule time         : 1.96408 usecs


===============================================================================

Packet I/O Totals:
  Received:      2121798
  Analyzed:       643495 ( 30.328%)
   Dropped:       618918 ( 22.582%)
  Filtered:            0 (  0.000%)
Outstanding:      1478303 ( 69.672%)
  Injected:            0


===============================================================================


About tunning sysctl options, if I am not wrong, OpenBSD tunes them
"on the fly" according to network load.

And more info: I have installed suricata in this host also to do more
tests, and suricata returns me best perfomance without losing many
packets:

-------------------------------------------------------------------
Counter                   | TM Name                   | Value
-------------------------------------------------------------------
capture.kernel_packets    | RxPcapem51                | 3052575199
capture.kernel_drops      | RxPcapem51                | 143259
capture.kernel_ifdrops    | RxPcapem51                | 0
decoder.pkts              | RxPcapem51                | 19561319
decoder.bytes             | RxPcapem51                | 15561225326
decoder.ipv4              | RxPcapem51                | 19561319
decoder.ipv6              | RxPcapem51                | 0
decoder.ethernet          | RxPcapem51                | 19561319
decoder.raw               | RxPcapem51                | 0
decoder.sll               | RxPcapem51                | 0
decoder.tcp               | RxPcapem51                | 19561139
decoder.udp               | RxPcapem51                | 0
decoder.sctp              | RxPcapem51                | 0
decoder.icmpv4            | RxPcapem51                | 180
decoder.icmpv6            | RxPcapem51                | 0
decoder.ppp               | RxPcapem51                | 0
decoder.pppoe             | RxPcapem51                | 0
decoder.gre               | RxPcapem51                | 0
decoder.vlan              | RxPcapem51                | 0
decoder.teredo            | RxPcapem51                | 0
decoder.ipv4_in_ipv6      | RxPcapem51                | 0
decoder.ipv6_in_ipv6      | RxPcapem51                | 0
decoder.avg_pkt_size      | RxPcapem51                | 796
decoder.max_pkt_size      | RxPcapem51                | 1506
defrag.ipv4.fragments     | RxPcapem51                | 0
defrag.ipv4.reassembled   | RxPcapem51                | 0
defrag.ipv4.timeouts      | RxPcapem51                | 0
defrag.ipv6.fragments     | RxPcapem51                | 0
defrag.ipv6.reassembled   | RxPcapem51                | 0
defrag.ipv6.timeouts      | RxPcapem51                | 0
defrag.max_frag_hits      | RxPcapem51                | 0
tcp.sessions              | Detect                    | 66702
tcp.ssn_memcap_drop       | Detect                    | 0
tcp.pseudo                | Detect                    | 7500
tcp.invalid_checksum      | Detect                    | 2
tcp.no_flow               | Detect                    | 0
tcp.reused_ssn            | Detect                    | 0
tcp.memuse                | Detect                    | 36175872
tcp.syn                   | Detect                    | 131466
tcp.synack                | Detect                    | 129929
tcp.rst                   | Detect                    | 56046
tcp.segment_memcap_drop   | Detect                    | 0
tcp.stream_depth_reached  | Detect                    | 306
tcp.reassembly_memuse     | Detect                    | 69060696
tcp.reassembly_gap        | Detect                    | 3214
detect.alert              | Detect                    | 38
flow_mgr.closed_pruned    | FlowManagerThread         | 78944
flow_mgr.new_pruned       | FlowManagerThread         | 3978
flow_mgr.est_pruned       | FlowManagerThread         | 2390
flow.memuse               | FlowManagerThread         | 3852512
flow.spare                | FlowManagerThread         | 10000
flow.emerg_mode_entered   | FlowManagerThread         | 0
flow.emerg_mode_over      | FlowManagerThread         | 0

Relevant data here are tcp.reassembly_gap and tcp.invalid_checksum

numbers.


Any idea please??



------------------------------------------------------------------------------

Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest

Snort news!



------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. A cloud service to automate IT design, transition and operations
2. Dashboards that offer high-level views of enterprise services
3. A single system of record for all IT processes
http://p.sf.net/sfu/servicenow-d2d-j
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest
Snort news!


------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. A cloud service to automate IT design, transition and operations
2. Dashboards that offer high-level views of enterprise services
3. A single system of record for all IT processes
http://p.sf.net/sfu/servicenow-d2d-j
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!
Previous By Date Next
Previous By Thread Next

Current thread: