Re: Snort inline extremely slow packet forwarding

[Hussein Bahaidarah <husseinb () gmail com>]

Thanks Rmkml for help,

I found a work around and I don't understand how and why it did work. 
First, let me explain my configuration:
eth2 and eth3 are bridged and snort IP should run on them
eth1 is not used

when I use:  "snort    -N -K none -k notcp -c rules/inline -A console   --daq afpacket -i eth3:eth2   -Q"  the slowness 
problem appear

my work around is to use " snort    -N -K none -k notcp -c rules/inline -A console   --daq afpacket -i eth3:eth1   -Q 
". This works fine though eth1 is not used!! 

With this now everything works fine and I can reach upto 350mbps with one rule only. I will start putting more rules 
and do stress testing with IXIA. However, I wish to understand the reasons behind this behavior. BTY, I did performance 
debug and I found that packet processing is very fast in both cases:

PPM: Process-BeginPkt[1591] caplen=1514
PPM: Pkt[1591] Used= 3.37051 usecs
PPM: Process-EndPkt[1591]

===============================================================================
Packet Performance Summary:
   max packet time       : 250 usecs
   packet events         : 0
   avg pkt time          : 4.62883 usecs
===============================================================================

I believe it might be a bug as the "outstanding shows huge number:
===============================================================================
Packet I/O Totals:
   Received:         1614
   Analyzed:         1617 (100.186%)
    Dropped:            0 (  0.000%)
   Filtered:            0 (  0.000%)
Outstanding: 18446744073709551613 (1142920946326490240.000%)
   Injected:            0
===============================================================================

 
On Jul 15, 2011, at 8:47 PM, rmkml wrote:

another link:
mikelococo.com/files/2011/2011_01_25-snort_performance.pdf


On Fri, 15 Jul 2011, rmkml wrote:

> another idea:
> http://www.gamelinux.org/?page_id=284
>
>
> On Fri, 15 Jul 2011, rmkml wrote:
>
> > ok,
> > do you have tested enlarge daq buffer like ?
> > http://seclists.org/snort/2011/q1/705
> > (it's freebsd plateform, but daq buffer is same on linux)
> >
> > another idea: search a linux distrib contains already snort v2.9.0.5 compiled?
> > Regards
> > Rmkml
> >
> >
> > On Fri, 15 Jul 2011, Hussein Bahaidarah wrote:
> >
> > > Hi Rmkml,
> > >
> > > I have tested bpf filter and removing the pass. I guess I might need to go back v2.8.6.1 as you said.
> > >
> > > Thanks,
> > >
> > > On Jul 15, 2011, at 8:26 PM, rmkml wrote:
> > >
> > > ok good,
> > > maybe try old snort like v2.8.6.1 with iptables/netfilter nfq ? (ips/inline mode)
> > > do you have tested with remove pass rule please?
> > > another test with bpf filter on snort.conf v291beta/daq?
> > > Regards
> > > Rmkml
> > >
> > >
> > > On Fri, 15 Jul 2011, Hussein Bahaidarah wrote:
> > >
> > > > Hi Rmkml,
> > > > CentOS is i386. Before starting with IXIA, I have done simple test which is doing ping and web browsing across the 
> > > > IPS
> > > > and it is very slow and disconnecting. I believe it is not a CPU or memory issue. Before starting the IPS, the 
> > > > linux
> > > > bridging is working fine giving me a speed of 300mbps when I test with speedtest.net. after running IPS,
> > > > the speed test can't go through and even ping starts dropping. BTY, when I use "--daq dump", it works very fast;
> > > > but of course "drop" does not work as Snort becomes in passive mode.
> > > > Thanks,
> > > >
> > > > On Jul 15, 2011, at 8:04 PM, rmkml wrote:
> > > >
> > > > thx you again Hussein,
> > > > no pb with last snort beta, but it's a beta...
> > > > (http://cvs.snort.org/viewcvs.cgi/snort/doc/README.counts?rev=1.2&sortby=log&content-type=text/vnd.viewcvs-markup)
> > > > what it's centos version please? i386? x86_64?
> > > > maybe first use common plateform like i386 and not last last linux version...
> > > > found a "stable" perform is complicated...
> > > > maybe it's a kernel pb, a libpcap pb, daq or snort of coursethis is why test with iptables/netfilter before...
> > > > (iptables bridge testing it's easy, and if you have similar pb...)
> > > > how can you test (without ixia) please? you like web surf through centos/snort plateform? and it's always slow 
> > > > performance? or it appear during specific (download/upload) test?
> > > > look netstat interface errors/stats, cpu, top, buffer during bench/test....
> > > > good luck
> > > > Rmkml
> > > >
> > > >
> > > >
> > > > On Fri, 15 Jul 2011, Hussein Bahaidarah wrote:
> > > >
> > > > > Hi Rmkml,
> > > > > the server is HP with 8GB and 8 CPU's. Running on CentOS and no VMWARE is involved.
> > > > > I could not run it with iptables as I get error loading it with "--daq nfq ".
> > > > > I have IXIA; but I need first Snort inline to work properly before using IXIA. I am using Snort 2.9.1 beta, would 
> > > > > this be the source of the problem?
> > > > > Thanks,
> > > > >
> > > > > On Jul 15, 2011, at 7:37 PM, rmkml wrote:
> > > > >
> > > > > Thx you Hussein,
> > > > > Maybe look https://www.procyonlabs.com/snort_manual/2.9/node7.html please
> > > > > Could you describe hardware please? vmware use?
> > > > > what is performance without snort but with only use iptables please?
> > > > > Note: snort output don't indicate drop packets...
> > > > > you don't need pass rule, try without please (normally no performance impact, just try please).
> > > > > how can you test performance please? spirent? breakingpoint? ixia? real internet trafic? (your test use very low 
> > > > > packets number in 20mn, do you have tested last snort GA like v2.9.0.5 please?)
> > > > > Regards
> > > > > Rmkml
> > > > >
> > > > >
> > > > > On Fri, 15 Jul 2011, Hussein Bahaidarah wrote:
> > > > >
> > > > > > Hi Rmkml,
> > > > > >
> > > > > > I had only one active rule at a time as I have just started testing the functionality of Snort IPS. the 
> > > > > > configuration/rule file is as below (with one pass rule active and the drop rules inactive):
> > > > > > [root@IPS snort]# vi rules/inline
> > > > > >
> > > > > > var LIB_PATH /usr/lib64
> > > > > > var CONF_PATH /etc/snort
> > > > > > var RULE_PATH $CONF_PATH/rules
> > > > > > var SO_RULE_PATH $CONF_PATH/so_rules
> > > > > > var PREPROC_RULE_PATH $CONF_PATH/preproc_rules
> > > > > >
> > > > > > config disable_decode_alerts
> > > > > > config disable_tcpopt_experimental_alerts
> > > > > > config disable_tcpopt_obsolete_alerts
> > > > > > config disable_tcpopt_ttcp_alerts
> > > > > > config disable_tcpopt_alerts
> > > > > > config disable_ipopt_alerts
> > > > > > config checksum_mode: noip notcp
> > > > > > config detection: search-method ac-bnfa split-any-any search-optimize no_stream_inserts
> > > > > > config event_queue: max_queue 1 log 1 order_events content_length
> > > > > >
> > > > > > #preprocessor stream5_global: max_tcp 2000,  track_udp no, track_icmp yes, track_tcp no max_active_responses 1 
> > > > > > min_response_seconds 5 memcap 32768
> > > > > > #preprocessor stream5_tcp: policy windows, dont_store_large_packets,  \
> > > > > > #   overlap_limit 0, small_segments 0 bytes 0, timeout 30,   max_queued_bytes 1024, max_queued_segs 2\
> > > > > > #   ignore_any_rules,  \
> > > > > > #    ports server 80
> > > > > >
> > > > > > #preprocessor http_inspect: global iis_unicode_map unicode.map 1252 memcap 2304  max_gzip_mem 3276
> > > > > > #preprocessor http_inspect_server: server default \
> > > > > > #   inspect_uri_only \
> > > > > > #   ports { 80 } \
> > > > > > #    webroot no
> > > > > >
> > > > > > output log_null
> > > > > > output alert_full:/dev/null
> > > > > > output log_tcpdump:/dev/null
> > > > > > output alert_fast: alert.fast
> > > > > > include threshold.conf
> > > > > > #####################
> > > > > > config policy_mode:inline
> > > > > > #####################
> > > > > > pass ip any any -> any any (sid:1)
> > > > > > #drop tcp any any -> any 80 ( content:" xxxphone.de" ;   sid:2)
> > > > > > #drop tcp any any -> any 80 ( content:" 201.213.215.168" ; fast_pattern:only;  react; sid:3)
> > > > > > #pass ip any any -> any any (sid:20)
> > > > > > ~
> > > > > > ~
> > > > > > "rules/inline" 40L, 1565C
> > > > > >
> > > > > >
> > > > > > The snort statistics are below:
> > > > > >
> > > > > >
> > > > > > [root@IPS snort]# snort    -N -K none -k notcp -c rules/inline -A console   --daq afpacket -i eth3:eth2   -Q  
> > > > > > --daq-mode inline
> > > > > > Enabling inline operation
> > > > > > Running in IDS mode
> > > > > >
> > > > > >    --== Initializing Snort ==--
> > > > > > Initializing Output Plugins!
> > > > > > Initializing Preprocessors!
> > > > > > Initializing Plug-ins!
> > > > > > Parsing Rules file "rules/inline"
> > > > > > Detection:
> > > > > > Search-Method = AC-BNFA-Q
> > > > > > Split Any/Any group = enabled
> > > > > > Search-Method-Optimizations = enabled
> > > > > > Tagged Packet Limit: 256
> > > > > > Log directory = /var/log/snort
> > > > > >
> > > > > > +++++++++++++++++++++++++++++++++++++++++++++++++++
> > > > > > Initializing rule chains...
> > > > > > 1 Snort rules read
> > > > > > 1 detection rules
> > > > > > 0 decoder rules
> > > > > > 0 preprocessor rules
> > > > > > 1 Option Chains linked into 1 Chain Headers
> > > > > > 0 Dynamic rules
> > > > > > +++++++++++++++++++++++++++++++++++++++++++++++++++
> > > > > >
> > > > > > +-------------------[Rule Port Counts]---------------------------------------
> > > > > > |             tcp     udp    icmp      ip
> > > > > > |     src       0       0       0       0
> > > > > > |     dst       0       0       0       0
> > > > > > |     any       1       1       1       1
> > > > > > |      nc       0       0       0       1
> > > > > > |     s+d       0       0       0       0
> > > > > > +----------------------------------------------------------------------------
> > > > > >
> > > > > > +-----------------------[detection-filter-config]------------------------------
> > > > > > | memory-cap : 1048576 bytes
> > > > > > +-----------------------[detection-filter-rules]-------------------------------
> > > > > > | none
> > > > > > -------------------------------------------------------------------------------
> > > > > >
> > > > > > +-----------------------[rate-filter-config]-----------------------------------
> > > > > > | memory-cap : 1048576 bytes
> > > > > > +-----------------------[rate-filter-rules]------------------------------------
> > > > > > | none
> > > > > > -------------------------------------------------------------------------------
> > > > > >
> > > > > > +-----------------------[event-filter-config]----------------------------------
> > > > > > | memory-cap : 1048576 bytes
> > > > > > +-----------------------[event-filter-global]----------------------------------
> > > > > > +-----------------------[event-filter-local]-----------------------------------
> > > > > > | none
> > > > > > +-----------------------[suppression]------------------------------------------
> > > > > > | none
> > > > > > -------------------------------------------------------------------------------
> > > > > > Rule application order: activation->dynamic->pass->drop->sdrop->reject->alert->log
> > > > > > Verifying Preprocessor Configurations!
> > > > > >
> > > > > > [ Port Based Pattern Matching Memory ]
> > > > > > afpacket DAQ configured to inline.
> > > > > > Acquiring network traffic from "eth3:eth2".
> > > > > > Reload thread starting...
> > > > > > Reload thread started, thread 0xb7752b90 (6886)
> > > > > >
> > > > > >    --== Initialization Complete ==--
> > > > > >
> > > > > > ,,_     -*> Snort! <*-
> > > > > > o"  )~   Version 2.9.1_beta IPv6 GRE (Build 47)
> > > > > > ''''    By Martin Roesch & The Snort Team: http://www.snort.org/snort/snort-team
> > > > > >       Copyright (C) 1998-2011 Sourcefire, Inc., et al.
> > > > > >       Using libpcap version 1.1.1
> > > > > >       Using PCRE version: 6.6 06-Feb-2006
> > > > > >       Using ZLIB version: 1.2.3
> > > > > >
> > > > > > Commencing packet processing (pid=6886)
> > > > > > Decoding Ethernet
> > > > > > *** Caught Int-Signal
> > > > > > ===============================================================================
> > > > > > Run time for packet processing was 1242.798806 seconds
> > > > > > Snort processed 20367 packets.
> > > > > > Snort ran for 0 days 0 hours 20 minutes 42 seconds
> > > > > > Pkts/min:         1018
> > > > > > Pkts/sec:           16
> > > > > > ===============================================================================
> > > > > > Packet I/O Totals:
> > > > > > Received:        20367
> > > > > > Analyzed:        20367 (100.000%)
> > > > > > Dropped:            0 (  0.000%)
> > > > > > Filtered:            0 (  0.000%)
> > > > > > Outstanding:            0 (  0.000%)
> > > > > > Injected:            0
> > > > > > ===============================================================================
> > > > > > Breakdown by protocol (includes rebuilt packets):
> > > > > >    Eth:        20367 (100.000%)
> > > > > >   VLAN:            0 (  0.000%)
> > > > > >    IP4:        18038 ( 88.565%)
> > > > > >   Frag:            0 (  0.000%)
> > > > > >   ICMP:        11038 ( 54.196%)
> > > > > >    UDP:          106 (  0.520%)
> > > > > >    TCP:         6894 ( 33.849%)
> > > > > >    IP6:            0 (  0.000%)
> > > > > > IP6 Ext:            0 (  0.000%)
> > > > > > IP6 Opts:            0 (  0.000%)
> > > > > >  Frag6:            0 (  0.000%)
> > > > > >  ICMP6:            0 (  0.000%)
> > > > > >   UDP6:            0 (  0.000%)
> > > > > >   TCP6:            0 (  0.000%)
> > > > > > Teredo:            0 (  0.000%)
> > > > > > ICMP-IP:            0 (  0.000%)
> > > > > >  EAPOL:            0 (  0.000%)
> > > > > > IP4/IP4:            0 (  0.000%)
> > > > > > IP4/IP6:            0 (  0.000%)
> > > > > > IP6/IP4:            0 (  0.000%)
> > > > > > IP6/IP6:            0 (  0.000%)
> > > > > >    GRE:            0 (  0.000%)
> > > > > > GRE Eth:            0 (  0.000%)
> > > > > > GRE VLAN:            0 (  0.000%)
> > > > > > GRE IP4:            0 (  0.000%)
> > > > > > GRE IP6:            0 (  0.000%)
> > > > > > GRE IP6 Ext:            0 (  0.000%)
> > > > > > GRE PPTP:            0 (  0.000%)
> > > > > > GRE ARP:            0 (  0.000%)
> > > > > > GRE IPX:            0 (  0.000%)
> > > > > > GRE Loop:            0 (  0.000%)
> > > > > >   MPLS:            0 (  0.000%)
> > > > > >    ARP:         1396 (  6.854%)
> > > > > >    IPX:            0 (  0.000%)
> > > > > > Eth Loop:          124 (  0.609%)
> > > > > > Eth Disc:            0 (  0.000%)
> > > > > > IP4 Disc:            0 (  0.000%)
> > > > > > IP6 Disc:            0 (  0.000%)
> > > > > > TCP Disc:            0 (  0.000%)
> > > > > > UDP Disc:            0 (  0.000%)
> > > > > > ICMP Disc:            0 (  0.000%)
> > > > > > All Discard:            0 (  0.000%)
> > > > > >  Other:          809 (  3.972%)
> > > > > > Bad Chk Sum:            0 (  0.000%)
> > > > > > Bad TTL:            0 (  0.000%)
> > > > > > S5 G 1:            0 (  0.000%)
> > > > > > S5 G 2:            0 (  0.000%)
> > > > > >  Total:        20367
> > > > > > ===============================================================================
> > > > > > Action Stats:
> > > > > > Alerts:            0 (  0.000%)
> > > > > > Logged:            0 (  0.000%)
> > > > > > Passed:        18038 ( 88.565%)
> > > > > > Limits:
> > > > > >  Match:            0
> > > > > >  Queue:        18038
> > > > > >    Log:            0
> > > > > >  Event:            0
> > > > > >  Alert:            0
> > > > > > Verdicts:
> > > > > >  Allow:        20367 (100.000%)
> > > > > >  Block:            0 (  0.000%)
> > > > > > Replace:            0 (  0.000%)
> > > > > > Whitelist:            0 (  0.000%)
> > > > > > Blacklist:            0 (  0.000%)
> > > > > > Ignore:            0 (  0.000%)
> > > > > > ===============================================================================
> > > > > > Snort exiting
> > > > > > [root@IPS snort]#
> > > > > >
> > > > > >
> > > > > > On Jul 15, 2011, at 6:01 PM, rmkml wrote:
> > > > > >
> > > > > > Hi Hussein,
> > > > > > maybe can you post snort output packet statistics to the list after few minutes/hours please?
> > > > > > can you post snort.conf? ok 1 drop sig, bute how many alert sig please?
> > > > > > Regards
> > > > > > Rmkml
> > > > > >
> > > > > >
> > > > > > On Fri, 15 Jul 2011, Hussein Bahaidarah wrote:
> > > > > >
> > > > > > > Hello,
> > > > > > > I am running snort 2.9.1 beta. it is extremely slow in packet forwarding though the rules file has 1 drop rule 
> > > > > > > only.
> > > > > > > The command line I am using is:
> > > > > > > snort    -N -K none -k notcp -c rules/inline -A console   --daq afpacket -i eth3:eth2   -Q  --daq-mode inline'
> > > > > > > Regards,
> > > > > > > Hussein
> > > > > > > ------------------------------------------------------------------------------


------------------------------------------------------------------------------
AppSumo Presents a FREE Video for the SourceForge Community by Eric 
Ries, the creator of the Lean Startup Methodology on "Lean Startup 
Secrets Revealed." This video shows you how to validate your ideas, 
optimize your ideas and identify your business strategy.
http://p.sf.net/sfu/appsumosfdev2dev
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users

Please see http://www.snort.org/docs for documentation