Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3

["C. L. Martinez" <carlopmart () gmail com>]

On Wed, Jun 12, 2013 at 4:15 PM, Victor Roemer <vroemer () sourcefire com> wrote:
> First, I would recommend you tune your policy down. I believe for most
> environments that this is not a
> quick process; something to keep in mind.
>
> Things to think about
>
> preprocessor pop ...
> preprocessor imap ...
> preprocessor smtp ...
>
> are each configured with
>
>     b64_decode_depth 0
>     qp_decode_depth 0
>     bitenc_decode_depth 0
>     uu_decode_depth 0
>
> which permits unlimited decoding of the specified encoding. If your seeing
> alot of email traffic,
> you may do better to assign a fixed decoding depth.
>
> Choosing an appropriate value that provides sufficient detection and
> performance is where tuning gets tricky.
>
> Similarly, if you don't need these preproc's you could also consider
> disabling them.
> (e.g., little traffic ever seen; or maybe you're rule-set doesn't benefit
> with them enabled).
>
>
> -------
>
> preprocessor ssh ...
>
> Without the appropriate preproc rules enabled, this preprocessor is not
> likely to be doing anything for you.
>
> You're also using the "autodetect" keyword in this preprocessor, which is a
> performance
> consideration as it will cause the preproc to look at traffic on all tcp
> ports.
>
>
> --------
>
>
> preprocessor http_inspect_server:
> ...
>     server_flow_depth 0
>     client_flow_depth 0
>
> Saving the best for last, HTTP Inspect will normally analyze the most
> traffic of all the
> application-level preprocs. The included options are easily the best place
> to start tuning here.
>
>
> --------
>
> Also curious why you have 2 output-plugins configured, this is not helping
> performance either.
> Its recommended to use unified2, especially if you want to be fast.
>
>
>
>
>
> Another thing to consider, and this should directly relate to Suricata
> getting better numbers,
> is load-balancing across multiple Snort processes.
>
> Though I may be mistaken, Suricata should be doing across multiple threads,
> rather than processes. (Anyone care to chime in?)


Thanks Victor for your help, I will try to adjust all preprocs but
last week I did another test and the results do not improve.

See this:

Rule application order:
activation->dynamic->pass->drop->sdrop->reject->alert->log
Verifying Preprocessor Configurations!
ICMP tracking disabled, no ICMP sessions allocated
IP tracking disabled, no IP sessions allocated
WARNING: flowbits key 'bpbrokenssl' is checked but not ever set.
1 out of 1024 flowbits in use.
pcap DAQ configured to passive.
Acquiring network traffic from "em4".
Reload thread starting...
Reload thread started, thread 0x74314dc300 (24837)
Decoding Ethernet

        --== Initialization Complete ==--

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.4.6 GRE (Build 73)
   ''''    By Martin Roesch & The Snort Team:
http://www.snort.org/snort/snort-team
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.3.0
           Using PCRE version: 8.31 2012-07-06
           Using ZLIB version: 1.2.3

           Rules Engine: SF_SNORT_DETECTION_ENGINE  Version 1.17  <Build 18>
           Rules Object: web-misc  Version 1.0  <Build 1>
           Rules Object: web-iis  Version 1.0  <Build 1>
           Rules Object: web-client  Version 1.0  <Build 1>
           Rules Object: web-activex  Version 1.0  <Build 1>
           Rules Object: specific-threats  Version 1.0  <Build 1>
           Rules Object: snmp  Version 1.0  <Build 1>
           Rules Object: smtp  Version 1.0  <Build 1>
           Rules Object: p2p  Version 1.0  <Build 1>
           Rules Object: nntp  Version 1.0  <Build 1>
           Rules Object: netbios  Version 1.0  <Build 1>
           Rules Object: multimedia  Version 1.0  <Build 1>
           Rules Object: misc  Version 1.0  <Build 1>
           Rules Object: imap  Version 1.0  <Build 1>
           Rules Object: icmp  Version 1.0  <Build 1>
           Rules Object: exploit  Version 1.0  <Build 1>
           Rules Object: dos  Version 1.0  <Build 1>
           Rules Object: chat  Version 1.0  <Build 1>
           Rules Object: bad-traffic  Version 1.0  <Build 1>
           Preprocessor Object: SF_DNP3  Version 1.1  <Build 1>
           Preprocessor Object: SF_MODBUS  Version 1.1  <Build 1>
           Preprocessor Object: SF_GTP  Version 1.1  <Build 1>
           Preprocessor Object: SF_REPUTATION  Version 1.1  <Build 1>
           Preprocessor Object: SF_SIP  Version 1.1  <Build 1>
           Preprocessor Object: SF_SDF  Version 1.1  <Build 1>
           Preprocessor Object: SF_DCERPC2  Version 1.0  <Build 3>
           Preprocessor Object: SF_SSLPP  Version 1.1  <Build 4>
           Preprocessor Object: SF_DNS  Version 1.1  <Build 4>
           Preprocessor Object: SF_SSH  Version 1.1  <Build 3>
           Preprocessor Object: SF_SMTP  Version 1.1  <Build 9>
           Preprocessor Object: SF_IMAP  Version 1.0  <Build 1>
           Preprocessor Object: SF_POP  Version 1.0  <Build 1>
           Preprocessor Object: SF_FTPTELNET  Version 1.2  <Build 13>
Commencing packet processing (pid=24837)
^C*** Caught Int-Signal
===============================================================================
Run time for packet processing was 134.291031 seconds
Snort processed 338306 packets.
Snort ran for 0 days 0 hours 2 minutes 14 seconds
   Pkts/min:       169153
   Pkts/sec:         2524
===============================================================================
Packet I/O Totals:
   Received:       962863
   Analyzed:       338306 ( 35.135%)
    Dropped:       367469 ( 27.622%)
   Filtered:            0 (  0.000%)
Outstanding:       624557 ( 64.865%)
   Injected:            0
===============================================================================

Here is the snort.test conf file I used:

#--------------------------------------------------
#   VRT Rule Packages Snort.conf
#
#   For more information visit us at:
#     http://www.snort.org                   Snort Website
#     http://vrt-blog.snort.org/    Sourcefire VRT Blog
#
#     Mailing list Contact:      snort-sigs () lists sourceforge net
#     False Positive reports:    fp () sourcefire com
#     Snort bugs:                bugs () snort org
#
#     Compatible with Snort Versions:
#     VERSIONS : 2.9.4.6
#
#     Snort build options:
#     OPTIONS : --enable-gre --enable-mpls --enable-targetbased
--enable-ppm --enable-perfprofiling --enable-zlib
--enable-active-response --enable-normalizer --enable-reload
--enable-react --enable-flexresp3
#
#     Additional information:
#     This configuration file enables active response, to run snort in
#     test mode -T you are required to supply an interface -i <interface>
#     or test mode will fail to fully validate the configuration and
#     exit with a FATAL error
#--------------------------------------------------

###################################################
# This file contains a sample snort configuration.
# You should take the following steps to create your own custom configuration:
#
#  1) Set the network variables.
#  2) Configure the decoder
#  3) Configure the base detection engine
#  4) Configure dynamic loaded libraries
#  5) Configure preprocessors
#  6) Configure output plugins
#  7) Customize your rule set
#  8) Customize preprocessor and decoder rule set
#  9) Customize shared object rule set
###################################################

###################################################
# Step #1: Set the network variables.  For more information, see
README.variables
###################################################

# Setup the network addresses you are protecting
ipvar HOME_NET 10.196.0.0/24

# Set up the external network addresses. Leave as "any" in most situations
ipvar EXTERNAL_NET !$HOME_NET

# List of DNS servers on your network
ipvar DNS_SERVERS $HOME_NET

# List of SMTP servers on your network
ipvar SMTP_SERVERS $HOME_NET

# List of web servers on your network
ipvar HTTP_SERVERS $HOME_NET

# List of sql servers on your network
ipvar SQL_SERVERS $HOME_NET

# List of telnet servers on your network
ipvar TELNET_SERVERS $HOME_NET

# List of ssh servers on your network
ipvar SSH_SERVERS $HOME_NET

# List of ftp servers on your network
ipvar FTP_SERVERS $HOME_NET

# List of sip servers on your network
ipvar SIP_SERVERS $HOME_NET

# List of ports you run web servers on
portvar HTTP_PORTS
[80,81,82,83,84,85,86,87,88,89,311,383,591,593,631,901,1220,1414,1741,1830,2301,2381,2809,3037,3128,3702,4343,4848,5250,6988,7000,7001,7144,7145,7510,7777,7779,8000,8008,8014,8028,8080,8085,8088,8090,8118,8123,8180,8181,8222,8243,8280,8300,8500,8800,8888,8899,9000,9060,9080,9090,9091,9443,9999,55555,1090,3200,3210,3300,3310,3333,3600,3610,8100,10000,11371,34443,34444,41080,50000:50010,51000:51010]

# List of ports you want to look for SHELLCODE on.
portvar SHELLCODE_PORTS !80

# List of ports you might see oracle attacks on
portvar ORACLE_PORTS 1024:

# List of ports you want to look for SSH connections on:
portvar SSH_PORTS 22

# List of ports you run ftp servers on
portvar FTP_PORTS [21,2100,3535]

# List of ports you run SIP servers on
portvar SIP_PORTS [5060,5061,5600]

# List of file data ports for file inspection
portvar FILE_DATA_PORTS [$HTTP_PORTS,110,143]

# List of GTP ports for GTP preprocessor
portvar GTP_PORTS [2123,2152,3386]

# other variables, these should not be modified
ipvar AIM_SERVERS
[64.12.24.0/23,64.12.28.0/23,64.12.161.0/24,64.12.163.0/24,64.12.200.0/24,205.188.3.0/24,205.188.5.0/24,205.188.7.0/24,205.188.9.0/24,205.188.153.0/24,205.188.179.0/24,205.188.248.0/24]

# Path to your rules files (this can be a relative path)
# Note for Windows users:  You are advised to make this an absolute path,
# such as:  c:\snort\rules
var RULE_PATH rules
var SO_RULE_PATH so_rules
#var PREPROC_RULE_PATH ../preproc_rules

# If you are using reputation preprocessor set these
# Currently there is a bug with relative paths, they are relative to
where snort is
# not relative to snort.conf like the above variables
# This is completely inconsistent with how other vars work, BUG 89986
# Set the absolute path appropriately
var WHITE_LIST_PATH iplists
var BLACK_LIST_PATH iplists

###################################################
# Step #2: Configure the decoder.  For more information, see README.decode
###################################################

# Stop generic decode events:
config disable_decode_alerts

# Stop Alerts on experimental TCP options
config disable_tcpopt_experimental_alerts

# Stop Alerts on obsolete TCP options
config disable_tcpopt_obsolete_alerts

# Stop Alerts on T/TCP alerts
config disable_tcpopt_ttcp_alerts

# Stop Alerts on all other TCPOption type events:
config disable_tcpopt_alerts

# Stop Alerts on invalid ip options
config disable_ipopt_alerts

# Alert if value in length field (IP, TCP, UDP) is greater th elength
of the packet
# config enable_decode_oversized_alerts

# Same as above, but drop packet if in Inline mode (requires
enable_decode_oversized_alerts)
# config enable_decode_oversized_drops

# Configure IP / TCP checksum mode
config checksum_mode: all

# Configure maximum number of flowbit references.  For more
information, see README.flowbits
# config flowbits_size: 64

# Configure ports to ignore
# config ignore_ports: tcp 21 6667:6671 1356
# config ignore_ports: udp 1:17 53

# Configure active response for non inline operation. For more
information, see REAMDE.active
# config response: eth0 attempts 2

# Configure DAQ related options for inline operation. For more
information, see README.daq
#
# config daq: <type>
# config daq_dir: <dir>
# config daq_mode: <mode>
# config daq_var: <var>
#
# <type> ::= pcap | afpacket | dump | nfq | ipq | ipfw
# <mode> ::= read-file | passive | inline
# <var> ::= arbitrary <name>=<value passed to DAQ
# <dir> ::= path as to where to look for DAQ module so's

config daq: pcap
config daq_dir: /data/soft/daq/lib/daq
config daq_mode: passive
config daq_var: buffer_size=384MB

# Configure specific UID and GID to run snort as after dropping privs.
For more information see snort -h command line options
#
# config set_gid:
# config set_uid:

# Configure default snaplen. Snort defaults to MTU of in use
interface. For more information see README
#
# config snaplen:
#

# Configure default bpf_file to use for filtering what traffic reaches
snort. For more information see snort -h command line options (-F)
#
# config bpf_file:
#

# Configure default log directory for snort to log to.  For more
information see snort -h command line options (-l)
#
config logdir: /nsm/sensor_data/idpsnort01


###################################################
# Step #3: Configure the base detection engine.  For more information,
see  README.decode
###################################################

# Configure PCRE match limitations
config pcre_match_limit: 3500
config pcre_match_limit_recursion: 1500

# Configure the detection engine  See the Snort Manual, Configuring
Snort - Includes - Config
config detection: search-method ac-split search-optimize max-pattern-len 20

# Configure the event queue.  For more information, see README.event_queue
config event_queue: max_queue 8 log 5 order_events content_length

# Configure protocol aware flushing
# For more information see README.stream5
###################################################
config paf_max: 16000

###################################################
# Step #4: Configure dynamic loaded libraries.
# For more information, see Snort Manual, Configuring Snort - Dynamic Modules
###################################################

# path to dynamic preprocessor libraries
dynamicpreprocessor directory /data/soft/snort/lib/snort_dynamicpreprocessor

# path to base preprocessor engine
dynamicengine /data/soft/snort/lib/snort_dynamicengine/libsf_engine.so

# path to dynamic rules libraries
#dynamicdetection directory /usr/local/lib/snort_dynamicrules
dynamicdetection directory /data/config/etc/idpsnort01/so_rules

###################################################
# Step #5: Configure preprocessors
# For more information, see the Snort Manual, Configuring Snort - Preprocessors
###################################################

# GTP Control Channle Preprocessor. For more information, see README.GTP
# preprocessor gtp: ports { 2123 3386 2152 }

# Inline packet normalization. For more information, see README.normalize
# Does nothing in IDS mode
#preprocessor normalize_ip4
#preprocessor normalize_tcp: ips ecn stream
#preprocessor normalize_icmp4
#preprocessor normalize_ip6
#preprocessor normalize_icmp6

# Target-based IP defragmentation.  For more inforation, see README.frag3
preprocessor frag3_global: max_frags 65536, memcap 33554432, prealloc_frags 8192
preprocessor frag3_engine: policy windows detect_anomalies
overlap_limit 10 min_fragment_length 100 timeout 180

# Target-Based stateful inspection/stream reassembly.  For more
inforation, see README.stream5
preprocessor stream5_global: memcap 134217728, track_tcp yes, \
   track_udp yes, \
   track_icmp no, \
   max_tcp 262144, \
   max_udp 131072, \
   max_active_responses 2, \
   min_response_seconds 5
preprocessor stream5_tcp: policy windows, detect_anomalies, require_3whs 180, \
   overlap_limit 10, small_segments 3 bytes 150, timeout 180,
max_queued_bytes 10485760, max_queued_segs 10485760, \
    ports client 21 22 23 25 42 53 70 79 109 110 111 113 119 135 136
137 139 143 \
        161 445 513 514 587 593 691 1433 1521 1741 2100 3306 6070 6665
6666 6667 6668 6669 \
        7000 8181 32770 32771 32772 32773 32774 32775 32776 32777 32778 32779, \
    ports both 80 81 82 83 84 85 86 87 88 89 110 311 383 443 465 563
591 593 631 636 901 989 992 993 994 995 1220 1414 1830 2301 2381 2809
3037 3128 3702 4343 4848 5250 6988 7907 7000 7001 7144 \
        7145 7510 7802 7777 7779 7801 7900 7901 7902 7903 7904 7905
7906 7908 7909 7910 7911 7912 7913 7914 7915 7916 \
        7917 7918 7919 7920 8000 8008 8014 8028 8080 8085 8088 8090
8118 8123 8180 8222 8243 8280 8300 8500 8800 8888 8899 9000 9060 9080
9090 9091 9443 9999 55555 \
        1090 3200 3210 3300 3310 3333 3600 3610 8100 10000 11371 34443
34444 41080 50000:50010 51000:51010
preprocessor stream5_udp: timeout 180


About using OpenBSD's libpcap. Can I modify something to force daq and
snort to use OpenBSD's libpcap??

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!