Re: Sensor overload - Too much traffic for Snort box?

[Matthew Watchinski <mwatchinski () sourcefire com>]

A couple other things to try.

Change search method to "ac-bnfa"
Set the memcap for stream4 higher than the default.
Switch off mysql and go to unified, then use barnyard to insert to mysql.

If that and the other suggestions on interface parameters don't get you
back up to speed enable ruleprofiling and start turning off rules with
really high time ticks.

Cheers,
-matt

Ray H. wrote:
> Having some trouble with dropped packets. Wondering if my snort box is under
> powered or if I have my monitor session setup incorrectly, or something I'm
> just overlooking.
>
> Any help would be greatly appreciated. I've tried to include all relevant
> information pertaining to my issue with dropped packets.
>
>
> V/r,
>
> Ray H.
>
>
>
>
> ========================================================================
> Hardware
>  
> Dell Optiplex GX620, RedHat Enterprise 5 ES
> 2GB RAM, Pentium Core2 Duo 3GHz, 7,200RPM 80GB SATA
> ETH0 = Onboard Broadcom (Management NIC)
> ETH1 = Netgear 10/100/1000 (ifconfig eth1 up promisc on boot)
> ETH1 on Cisco 4506 Gigabit blade
> Receiving monitor session vlan 1-5 traffic
> ========================================================================
> ========================================================================
> ========================================================================
> snort-2.6.1.5 compiled with
>  
> ./configure --enable-dynamicplugin --enable-timestats --enable-perfprofiling
> --enable-linux-smp-stats --enable-gre --with-mysql
>
> Started with
>
> /usr/local/bin/snort -qc /etc/snort/snort.conf -i eth1 -D
> ========================================================================
> snort.conf
>  
> var HOME_NET
> [1.8.1.0/24,2.2.2.0/24,4.4.4.0/22,1.7.9.0/24,2.2.8.0/24,1.9.1.0/22,1.9.5.0/2
> 4] (IP's changed obviously)
> var EXTERNAL_NET !$HOME_NET
> var DNS_SERVERS 2.2.1.7
> var SMTP_SERVERS 2.2.1.2
> var HTTP_SERVERS $HOME_NET
> var SQL_SERVERS $HOME_NET
> var TELNET_SERVERS $HOME_NET
> var SNMP_SERVERS $HOME_NET
> var HTTP_PORTS 80 443
> var SSH_PORTS 22
> var RPC_PORTS 138 139 445
> var SHELLCODE_PORTS !80
> var ORACLE_PORTS 1521
> var RULE_PATH /etc/snort/rules
> config disable_decode_alerts
> config disable_tcpopt_experimental_alerts
> dynamicpreprocessor directory /usr/local/lib/snort_dynamicpreprocessor/
> dynamicengine /usr/local/lib/snort_dynamicengine/libsf_engine.so
> preprocessor perfmonitor: time 60 file /var/log/snort/perfmon.txt pktcnt 500
> preprocessor flow: stats_interval 0 hash 2
> preprocessor frag3_global: max_frags 65536
> preprocessor frag3_engine: policy first detect_anomalies
> preprocessor stream4: disable_evasion_alerts
> preprocessor stream4_reassemble
> preprocessor http_inspect: global iis_unicode_map unicode.map 1252
> preprocessor http_inspect_server: server default profile all ports { 80 8080
> 8180 } oversize_dir_length 500 no_alerts
> preprocessor rpc_decode: 111 32771
> preprocessor bo
> preprocessor ftp_telnet: global encrypted_traffic yes inspection_type
> stateful
> preprocessor ftp_telnet_protocol: telnet normalize ayt_attack_thresh 200
> preprocessor ftp_telnet_protocol: ftp server default def_max_param_len 100
> alt_max_param_len 200 { CWD } cmd_validity MODE < char ASBCZ > cmd_validity
> MDTM < [ date nnnnnnnnnnnnnn[.n[n[n]]] ] string > chk_str_fmt { USER PASS
> RNFR RNTO SITE MKD } telnet_cmds yes data_chan
> preprocessor ftp_telnet_protocol: ftp client default max_resp_len 256 bounce
> yes telnet_cmds yes
> preprocessor smtp: ports { 25 } inspection_type stateful normalize cmds
> normalize_cmds { EXPN VRFY RCPT } alt_max_command_line_len 260 { MAIL }
> alt_max_command_line_len 300 { RCPT } alt_max_command_line_len 500 { HELP
> HELO ETRN } alt_max_command_line_len 255 { EXPN VRFY }
>  
> output database: log, mysql, user=user password=password dbname=database
> host=localhost
>  
> include /etc/snort/local.rules
> include /etc/snort/bleeding-all.rules
> include $RULE_PATH/bad-traffic.rules
> include $RULE_PATH/exploit.rules
> include $RULE_PATH/scan.rules
> include $RULE_PATH/finger.rules
> include $RULE_PATH/ftp.rules
> include $RULE_PATH/telnet.rules
> include $RULE_PATH/rpc.rules
> include $RULE_PATH/rservices.rules
> include $RULE_PATH/dos.rules
> include $RULE_PATH/ddos.rules
> include $RULE_PATH/dns.rules
> include $RULE_PATH/tftp.rules
> include $RULE_PATH/web-cgi.rules
> include $RULE_PATH/web-coldfusion.rules
> include $RULE_PATH/web-iis.rules
> include $RULE_PATH/web-frontpage.rules
> include $RULE_PATH/web-misc.rules
> include $RULE_PATH/web-client.rules
> include $RULE_PATH/web-php.rules
> include $RULE_PATH/sql.rules
> include $RULE_PATH/x11.rules
> include $RULE_PATH/netbios.rules
> include $RULE_PATH/misc.rules
> include $RULE_PATH/attack-responses.rules
> include $RULE_PATH/oracle.rules
> include $RULE_PATH/mysql.rules
> include $RULE_PATH/smtp.rules
> include $RULE_PATH/imap.rules
> include $RULE_PATH/pop2.rules
> include $RULE_PATH/pop3.rules
> include $RULE_PATH/nntp.rules
> include $RULE_PATH/other-ids.rules
> include /etc/snort/threshold.conf
>
> ========================================================================
> /etc/snort/threshold.conf has 120 lines of rules
> ========================================================================
> ========================================================================
> ========================================================================
>
> tcpdump -n -i eth1 -s 1515 -w /root/tcpdump.pcap
> ** RUNS 5 MINUTES 3GB file created**
> 2,775,165 packets captured
> 6,094,867 packets received by filter
> 544,521 packets dropped by kernel
>
> ========================================================================
> ========================================================================
> ========================================================================
>
> iptraf results
> iface_stats_detailed-eth1.log
>  
>  
> Mon Jun  4 09:11:14 2007; 
> ******** Detailed interface statistics started ********
>  
> Detailed statistics for interface eth1, generated Mon Jun 4 09:11:25 2007
>  
> Total:  125,029 packets, 140,584,004 bytes
> (incoming: 125,029 packets, 140,584,004 bytes; outgoing: 0 packets, 0 bytes)
>
> IP:     125,029 packets, 138,730,999 bytes
> (incoming: 125,029 packets, 138,730,999 bytes; outgoing: 0 packets, 0 bytes)
>
> TCP: 124,064 packets, 138,595,840 bytes
> (incoming: 124,064 packets, 138,595,840 bytes; outgoing: 0 packets, 0 bytes)
>
> UDP: 646 packets, 91,865 bytes
> (incoming: 646 packets, 91,865 bytes; outgoing: 0 packets, 0 bytes)
>
> ICMP: 319 packets, 43,294 bytes
> (incoming: 319 packets, 43294 bytes; outgoing: 0 packets, 0 bytes)
>
>
> Broadcast: 21 packets, 1,932 bytes
>  
> Average rates:
>
> 12,480.82 kbytes/s, 11,366.27 packets/s
>
> Incoming: 12,480.82 kbytes/s, 11,366.27 packets/s
>
>  
> Peak total activity: 13,670.99 kbytes/s, 12,143.80 packets/s
>  
> IP checksum errors: 0
>  
> Running time: 11 seconds
> ========================================================================
> ========================================================================
> ========================================================================
>
>
> Detailed statistics for interface eth1, generated Mon Jun  4 15:13:28 2007
>  
> Total:  1,318,075 packets, 1,493,090,847 bytes
> (incoming: 1,318,075 packets, 1,493,090,847 bytes)
>
> IP:     1,318,075 packets, 1,473,611,296 bytes
>  (incoming: 1,318,075 packets, 1,473,611,296 bytes;)
>
> TCP: 1,310,898 packets, 1,472,524,935 bytes
> (incoming: 1,310,898 packets, 1,472,524,935 bytes)
>
> UDP: 5,628 packets, 942,292 bytes
> (incoming: 5628 packets, 942,292 bytes; outgoing: 0 packets, 0 bytes)
>
> ICMP: 1,549 packets, 144,069 bytes
> (incoming: 1,549 packets, 144,069 bytes; outgoing: 0 packets, 0 bytes)
> Broadcast: 257 packets, 34,332 bytes
>  
>
> Average rates:
>
>
> 12,150.80 kbytes/s, 10,983.96 packets/s
>
>  
> Peak total activity: 16,696.44 kbytes/s, 14,222.40 packets/s
>  
> IP checksum errors: 0
>  
> Running time: 120 seconds
>  
> ========================================================================
> ========================================================================
> ========================================================================
> snort.log
>  
> Jun 4 15:31:55: Snort ran for 0 Days 1 Hours 16 Minutes 25 Seconds
> Jun 4 15:31:55: Packet analysis time averages:
> Jun 4 15:31:55: Snort Analyzed 92,735,903 Packets Per Hour
> Jun 4 15:31:55: Snort Analyzed 1,220,209 Packets Per Minute
> Jun 4 15:31:55: Snort Analyzed 20,225 Packets Per Second
> Jun 4 15:31:55:
> Jun 4 15:31:55: Snort received 92,735,903 packets
> Jun 4 15:31:55:     Analyzed: 29,326,904(31.624%)
> Jun 4 15:31:55:     Dropped: 34,081,976(36.752%)
> Jun 4 15:31:55:     Outstanding: 29,327,023(31.624%)
> Jun 4 15:31:55:
> ========================================================================
> Jun 4 15:31:55: Breakdown by protocol:
> Jun 4 15:31:55:     TCP: 28,928,351   (98.639%)
> Jun 4 15:31:55:     UDP: 201,577      (0.687%)
> Jun 4 15:31:55:    ICMP: 61,033       (0.208%)
> Jun 4 15:31:55:     ARP: 14,381       (0.049%)
> Jun 4 15:31:55:   EAPOL: 0            (0.000%)
> Jun 4 15:31:55:    IPv6: 0            (0.000%)
> Jun 4 15:31:55: ETHLOOP: 808          (0.003%)
> Jun 4 15:31:55:     IPX: 510          (0.002%)
> Jun 4 15:31:55:     GRE: 0            (0.000%)
> Jun 4 15:31:55:    FRAG: 2,206        (0.008%)
> Jun 4 15:31:55:   OTHER: 119,029      (0.406%)
> Jun 4 15:31:55: DISCARD: 0            (0.000%)
> Jun 4 15:31:55:
> ========================================================================
> Jun 4 15:31:55: Action Stats:
> Jun 4 15:31:55: ALERTS: 613
> Jun 4 15:31:55: LOGGED: 613
> Jun 4 15:31:55: PASSED: 0
> Jun 4 15:31:55:
> ========================================================================
> Jun 4 15:31:55: Fragmentation Stats:
> Jun 4 15:31:55: Fragmented IP Packets: 2,206 (0.008%)
> Jun 4 15:31:55:     Fragment Trackers: 1,112
> Jun 4 15:31:55:    Rebuilt IP Packets: 541
> Jun 4 15:31:55:    Frag elements used: 0
> Jun 4 15:31:55: Discarded(incomplete): 0
> Jun 4 15:31:55:    Discarded(timeout): 0
> Jun 4 15:31:55:   Frag2 memory faults: 0
> Jun 4 15:31:55:
> ========================================================================
> Jun 4 15:31:55: TCP Stream Reassembly Stats:
> Jun 4 15:31:55:     TCP Packets Used: 28,928,200 (98.639%)
> Jun 4 15:31:55:     Stream Trackers: 223,097
> Jun 4 15:31:55:     Stream flushes: 861,589
> Jun 4 15:31:55:     Segments used: 2,059,808
> Jun 4 15:31:55:     Segments Queued: 2,207,190
> Jun 4 15:31:55:     Stream4 Memory Faults: 0
> Jun 4 15:31:55:
> ========================================================================
> Jun 4 15:31:55: HTTP Inspect - encodings (Note: stream-reassembled packets
> not normalized out):
> Jun 4 15:31:55:     POST methods: 17,156
> Jun 4 15:31:55:     GET methods: 319,091
> Jun 4 15:31:55:     Post parameters extracted: 58,368
> Jun 4 15:31:55:     Unicode: 35,401
> Jun 4 15:31:55:     Double unicode: 0
> Jun 4 15:31:55:     Non-ASCII representable: 436,642
> Jun 4 15:31:55:     Base 36: 0
> Jun 4 15:31:55:     Directory traversals: 4
> Jun 4 15:31:55:     Extra slashes ("//"): 34,143
> Jun 4 15:31:55:     Self-referencing paths ("./"):  4
> Jun 4 15:31:55:     Total packets processed: 20,766,980
> Jun 4 15:31:55:
> ========================================================================
> ========================================================================
> ========================================================================
>  
> Jun 4 08:52:07: Snort ran for 0 Days 0 Hours 27 Minutes 48 Seconds
> Jun 4 08:52:07: Packet analysis time averages:
> Jun 4 08:52:07: Snort Analyzed 1,197,427 Packets Per Minute
> Jun 4 08:52:07: Snort Analyzed 19,382 Packets Per Second
> Jun 4 08:52:07:
> Jun 4 08:52:07: Snort received 32,330,531 packets
> Jun 4 08:52:07:     Analyzed: 9,382,891(29.022%)
> Jun 4 08:52:07:     Dropped: 13,564,628(41.956%)
> Jun 4 08:52:07:     Outstanding: 9,383,012(29.022%)
> Jun 4 08:52:07:
> ========================================================================
> Jun 4 08:52:07: Breakdown by protocol:
> Jun 4 08:52:07:     TCP: 9,225,917    (98.326%)
> Jun 4 08:52:07:     UDP: 86,533       (0.922%)
> Jun 4 08:52:07:    ICMP: 22,799       (0.243%)
> Jun 4 08:52:07:     ARP: 4,861        (0.052%)
> Jun 4 08:52:07:   EAPOL: 0            (0.000%)
> Jun 4 08:52:07:    IPv6: 0            (0.000%)
> Jun 4 08:52:07: ETHLOOP: 298          (0.003%)
> Jun 4 08:52:07:     IPX: 196          (0.002%)
> Jun 4 08:52:07:     GRE: 0            (0.000%)
> Jun 4 08:52:07:    FRAG: 578          (0.006%)
> Jun 4 08:52:07:   OTHER: 41,997       (0.448%)
> Jun 4 08:52:07: DISCARD: 0            (0.000%)
> Jun 4 08:52:07:
> ========================================================================
> Jun 4 08:52:07: Action Stats:
> Jun 4 08:52:07: ALERTS: 173
> Jun 4 08:52:07: LOGGED: 173
> Jun 4 08:52:07: PASSED: 0
> Jun 4 08:52:07:
> ========================================================================
> Jun 4 08:52:07: Fragmentation Stats:
> Jun 4 08:52:07: Fragmented IP Packets: 578 (0.006%)
> Jun 4 08:52:07:     Fragment Trackers: 290
> Jun 4 08:52:07:    Rebuilt IP Packets: 141
> Jun 4 08:52:07:    Frag elements used: 0
> Jun 4 08:52:07: Discarded(incomplete): 0
> Jun 4 08:52:07:    Discarded(timeout): 0
> Jun 4 08:52:07:   Frag2 memory faults: 0
> Jun 4 08:52:07:
> ========================================================================
> Jun 4 08:52:07: TCP Stream Reassembly Stats:
> Jun 4 08:52:07:     TCP Packets Used: 9,225,853 (98.325%)
> Jun 4 08:52:07:     Stream Trackers: 57,701
> Jun 4 08:52:07:     Stream flushes: 272,567
> Jun 4 08:52:07:     Segments used: 622,016
> Jun 4 08:52:07:     Segments Queued: 661,535
> Jun 4 08:52:07:     Stream4 Memory Faults: 0
> Jun 4 08:52:07:
> ========================================================================
> Jun 4 08:52:07: HTTP Inspect - encodings (Note: stream-reassembled packets
> not normalized out):
> Jun 4 08:52:07:     POST methods: 7,001
> Jun 4 08:52:07:     GET methods: 110,973
> Jun 4 08:52:07:     Post parameters extracted: 20,367
> Jun 4 08:52:07:     Unicode: 4,222
> Jun 4 08:52:07:     Double unicode: 0
> Jun 4 08:52:07:     Non-ASCII representable: 90,762
> Jun 4 08:52:07:     Base 36: 0
> Jun 4 08:52:07:     Directory traversals: 0
> Jun 4 08:52:07:     Extra slashes ("//"): 13,083
> Jun 4 08:52:07:     Self-referencing paths ("./"):  0
> Jun 4 08:52:07:     Total packets processed: 6,616,832
> Jun 4 08:52:07:
> ========================================================================
> ========================================================================
> ========================================================================
>
> Jun 4 08:18:19: Snort ran for 2 Days 22 Hours 57 Minutes 34 Seconds
> Jun 4 08:18:19: Packet analysis time averages:
> Jun 4 08:18:19: Snort Analyzed 523,812,167 Packets Per Day
> Jun 4 08:18:19: Snort Analyzed 149,66,061 Packets Per Hour
> Jun 4 08:18:19: Snort Analyzed 246,094 Packets Per Minute
> Jun 4 08:18:19: Snort Analyzed 4,101 Packets Per Second
> Jun 4 08:18:19:
> Jun 4 08:18:19: Snort received 1,047,624,335 packets
> Jun 4 08:18:19:     Analyzed: 309,401,958 (29.534%)
> Jun 4 08:18:19:     Dropped: 428,820,298 (40.933%)
> Jun 4 08:18:19:     Outstanding: 309,402,079 (29.534%)
> Jun 4 08:18:19:
> ========================================================================
> Jun 4 08:18:19: Breakdown by protocol:
> Jun 4 08:18:19:     TCP: 290,576,825  (93.911%)
> Jun 4 08:18:19:     UDP: 8,327,653    (2.691%)
> Jun 4 08:18:19:    ICMP: 2,660,651    (0.860%)
> Jun 4 08:18:19:     ARP: 891,322     (0.288%)
> Jun 4 08:18:19:   EAPOL: 0          (0.000%)
> Jun 4 08:18:19:    IPv6: 24         (0.000%)
> Jun 4 08:18:19: ETHLOOP: 49,789      (0.016%)
> Jun 4 08:18:19:     IPX: 40,620      (0.013%)
> Jun 4 08:18:19:     GRE: 3          (0.000%)
> Jun 4 08:18:19:    FRAG: 68,260      (0.022%)
> Jun 4 08:18:19:   OTHER: 6,815,710    (2.203%)
> Jun 4 08:18:19: DISCARD: 0          (0.000%)
> Jun 4 08:18:19:
> ========================================================================
> Jun 4 08:18:19: Action Stats:
> Jun 4 08:18:19: ALERTS: 18,964
> Jun 4 08:18:19: LOGGED: 18,964
> Jun 4 08:18:19: PASSED: 0
> Jun 4 08:18:19:
> ========================================================================
> Jun 4 08:18:19: Fragmentation Stats:
> Jun 4 08:18:19: Fragmented IP Packets: 68,260 (0.022%)
> Jun 4 08:18:19:     Fragment Trackers: 34,216
> Jun 4 08:18:19:    Rebuilt IP Packets: 16,912
> Jun 4 08:18:19:    Frag elements used: 0
> Jun 4 08:18:19: Discarded(incomplete): 0
> Jun 4 08:18:19:    Discarded(timeout): 0
> Jun 4 08:18:19:   Frag2 memory faults: 0
> Jun 4 08:18:19:
> ========================================================================
> Jun 4 08:18:19: TCP Stream Reassembly Stats:
> Jun 4 08:18:19:     TCP Packets Used: 290,561,908 (93.906%)
> Jun 4 08:18:19:     Stream Trackers: 2,823,094
> Jun 4 08:18:19:     Stream flushes: 8,224,509
> Jun 4 08:18:19:     Segments used: 19,818,243
> Jun 4 08:18:19:     Segments Queued: 22,112,984
> Jun 4 08:18:19:     Stream4 Memory Faults: 0
> Jun 4 08:18:19:
> ========================================================================
> Jun 4 08:18:19: HTTP Inspect - encodings (Note:stream-reassembled packets
> not normalized out):
> Jun 4 08:18:19:     POST methods: 560,087
> Jun 4 08:18:19:     GET methods: 2,080,179
> Jun 4 08:18:19:     Post parameters extracted: 595,603
> Jun 4 08:18:19:     Unicode: 80,205
> Jun 4 08:18:19:     Double unicode: 0
> Jun 4 08:18:19:     Non-ASCII representable: 1,520,599
> Jun 4 08:18:19:     Base 36: 0
> Jun 4 08:18:19:     Directory traversals: 21,792
> Jun 4 08:18:19:     Extra slashes ("//"): 237,689
> Jun 4 08:18:19:     Self-referencing paths ("./"):  21,792
> Jun 4 08:18:19:     Total packets processed: 203,925,384
> Jun 4 08:18:19:
> ========================================================================
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users