Snort++: how to get multithreading to work?

["Prude, Terrell (SCC)" <Terrell_Prude () scc senate gov>]

Hello folks,

This is my first post.  We've been running "regular" Snort since the 2.9.5.x days and thought we'd give the new Snort 
3.0.0 Alpha a whirl.  For us, the major attraction to Snort++ is the multithreading for reasons of capacity.

Unfortunately, I'm having some trouble figuring out how to get that to work.  So far, the Snort process looks like it's 
still using only one CPU.    Snort itself seems to start right up and is "snorting" packets, and we are getting output 
in the Unified2 format.

Could someone point me in the right direction as to what I'm missing?


Platform:

------------------------------------
Processor:  Intel 4GHz quad-core w/ hyperthreading
DRAM:  32 GB
Disk space:  2TB, with about 1.9TB free
NIC for Snorting:  Intel X520-SR2 10Gbit fiber Ethernet
NIC for management:  Realtek 8169 built-in 1Gbit copper Ethernet
OS:  CentOS 7.1
Snort version:  3.0.0-a1-155
LuaJIT version:  2.0.4
DAQ version:  2.0.5
------------------------------------

All the ./configure stuff uses the default paths, i. e. the /usr/local tree.  I tried to stay as plain-vanilla as I 
reasonably could that way.

The configure statement:

------------------------------------
./configure --disable-silent-rules --enable-ppm --enable-perf-profiling --enable-large-pcap
------------------------------------

I then ran the make statement with -j8, per the directions, followed by "make install".  That looked good.


Command line to invoke Snort:

------------------------------------
/usr/local/bin/snort -D -i enp1s0f0 -c /usr/local/etc/snort/snort.lua -l /var/log/snort -z 8
------------------------------------


The log output from when Snort starts:

------------------------------------
Jun 17 04:07:47 p-its-idssnort2 snort[2984]: --------------------------------------------------
Jun 17 04:07:47 p-its-idssnort2 snort[2984]: o")~   Snort++ 3.0.0-a1-155
Jun 17 04:07:47 p-its-idssnort2 snort[2984]: --------------------------------------------------
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Loading /usr/local/etc/snort/snort.lua:
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: ips
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: active
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: classifications
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: rpc_decode
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream_tcp
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: binder
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: unified2
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream_ip
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: event_queue
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: detection
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: network
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: normalizer
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: references
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream_udp
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: search_engine
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Finished /usr/local/etc/snort/snort.lua.
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Loading rules:
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Loading /usr/local/etc/snort/rules/local.rules:
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Finished /usr/local/etc/snort/rules/local.rules.
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Finished rules.
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: --------------------------------------------------
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: rule counts
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: total rules loaded: 2304
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: text rules: 2304
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: option chains: 2304
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: chain headers: 2304
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: --------------------------------------------------
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: rule port counts
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: tcp     udp    icmp      ip
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: any    2304    2304    2304    2304
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: nc       0       0       0    2304
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: --------------------------------------------------
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: pcap DAQ configured to passive.
Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Initializing daemon mode
Jun 17 04:07:48 p-its-idssnort2 snort[2993]: Daemon initialized, signaled parent pid: 2984
Jun 17 04:07:48 p-its-idssnort2 snort[2993]: Writing PID "2993" to file "/var/log/snort/snort.pid"
Jun 17 04:07:48 p-its-idssnort2 snort[2993]: Commencing packet processing
Jun 17 04:07:48 p-its-idssnort2 snort[2993]: ++ [0] enp1s0f0
------------------------------------


The log output after I kill the Snort process:

------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ** caught term signal
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: == stopping
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: -- [0] enp1s0f0
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: Packet Statistics
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: daq
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: pcaps: 1
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: received: 1120415147
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: analyzed: 23673620
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: dropped: 1096740235
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: outstanding: 1096741527
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: allow: 23673620
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: idle: 1
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: codec
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: total: 23673624            (100.000%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: other: 555                 (  0.002%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: discards: 2609430          ( 11.023%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: auth: 769          (  0.003%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: esp: 211987        (  0.895%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: eth: 23673624      (100.000%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: gre: 8574          (  0.036%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: icmp4: 2671                (  0.011%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: icmp4_ip: 1277             (  0.005%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: icmp6: 26                  (  0.000%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ipv4: 23673624     (100.000%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ipv6: 58           (  0.000%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ipv6_no_next: 31           (  0.000%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ppp_encap: 8574            (  0.036%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp: 16224198      ( 68.533%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: teredo: 58                 (  0.000%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: udp: 4838675       ( 20.439%)
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: Module Statistics
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: bad checksum (ip4): 10418
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: binder
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: packets: 4399149
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: inspects: 4399149
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ip flows: 2800
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp flows: 4296173
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp prunes: 4165102
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: udp flows: 100176
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream_ip
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: fragments: 220
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: reassembled: 4
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers added: 216
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers freed: 216
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: nodes inserted: 220
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: nodes deleted: 220
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream_tcp
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: sessions: 4296173
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: discards: 170170
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: events: 3999594
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: syn trackers: 247856
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: syn-ack trackers: 6903
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: data trackers: 109476
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers created: 364235
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers released: 364235
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs queued: 335355
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs released: 335355
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs split: 227
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs used: 56291
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: rebuilt packets: 22755
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: rebuilt buffers: 42889
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: overlaps: 28
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: gaps: 73264
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: max segs: 15128
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: max bytes: 137382
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: client cleanups: 73351
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: server cleanups: 66854
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream_udp
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: sessions: 100176
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: created: 100176
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: released: 100176
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: Summary Statistics
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: detection
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: analyzed: 23673620
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: process
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: signals: 1
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: timing
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: runtime: 01:25:16
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: seconds: 5116.16403
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: packets: 23673620
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: pkts/sec: 4627
Jun 17 05:33:04 p-its-idssnort2 snort[2993]: o")~   Snort exiting
------------------------------------


The "top" output while Snort++ is running:

------------------------------------
top - 05:20:58 up  2:15,  3 users,  load average: 1.00, 1.01, 0.99
Tasks: 201 total,   1 running, 200 sleeping,   0 stopped,   0 zombie
%Cpu0  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  :  0.0 us,  0.0 sy,  0.0 ni, 99.7 id,  0.3 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu4  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu5  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu6  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu7  : 97.3 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  2.7 si,  0.0 st
KiB Mem : 32703168 total, 31865964 free,   659032 used,   178172 buff/cache
KiB Swap:     4092 total,     4092 free,        0 used. 31846588 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
2993 root      20   0  379776 327360   4040 S 100.0  1.0  73:23.35 snort
    1 root      20   0   56652   6728   3908 S   0.0  0.0   0:00.76 systemd
    2 root      20   0       0      0      0 S   0.0  0.0   0:00.00 kthreadd
    3 root      20   0       0      0      0 S   0.0  0.0   0:00.02 ksoftirqd/0
    5 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:0H
    7 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 migration/0
    8 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh
    9 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/0
   10 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/1
   11 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/2
   12 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/3
   13 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/4
   14 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/5
   15 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/6
   16 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/7
   17 root      20   0       0      0      0 S   0.0  0.0   0:00.22 rcu_sched
   18 root      20   0       0      0      0 S   0.0  0.0   0:00.09 rcuos/0
   19 root      20   0       0      0      0 S   0.0  0.0   0:00.08 rcuos/1
   20 root      20   0       0      0      0 S   0.0  0.0   0:00.01 rcuos/2
   21 root      20   0       0      0      0 S   0.0  0.0   0:00.01 rcuos/3
   22 root      20   0       0      0      0 S   0.0  0.0   0:00.03 rcuos/4
   23 root      20   0       0      0      0 S   0.0  0.0   0:00.01 rcuos/5
   24 root      20   0       0      0      0 S   0.0  0.0   0:00.03 rcuos/6
   25 root      20   0       0      0      0 S   0.0  0.0   0:00.01 rcuos/7
   26 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 watchdog/0
   27 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 watchdog/1
   28 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 migration/1
   29 root      20   0       0      0      0 S   0.0  0.0   0:00.01 ksoftirqd/1
------------------------------------

And finally, what the NIC itself is reporting for traffic that it's seeing.  We're seeing it come in, all right.  :)  
So far, no errors, collisions, or any other apparent nasties.

------------------------------------
$ ip -s link show enp1s0f0
3: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 90:e2:ba:85:28:74 brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast
    1865322070123 1892842032 0       0       0       8445
    TX: bytes  packets  errors  dropped carrier collsns
    0          0        0       0       0       0
------------------------------------


------------------------------------------------------------------------------

_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about Snort!