Re: snort3: problem with http_inspect

["Al Lewis (allewi)" <allewi () cisco com>]

If you only need to run the extra codecs you should be able to run it like this (point to the codecs directly):

ALLEWI-M-8257:marcin-issue allewi$ ./bin/snort -c etc/snort/marcin.lua 
--plugin-path=/var/tmp/marcin-issue/lib/snort_extra/codecs -r ~/Downloads/marcin-sent.pcap -Acsv -q
02/26-08:19:45.017007, 4, TCP, raw, 133, C2S, 192.168.17.20:34616, 192.168.17.30:80, 1:4000003:0, allow
02/26-08:19:45.017007, 5, TCP, stream_tcp, 57, C2S, 192.168.17.20:34616, 192.168.17.30:80, 1:3000002:0, allow
02/26-08:19:45.017007, 5, TCP, stream_tcp, 57, C2S, 192.168.17.20:34616, 192.168.17.30:80, 1:3000001:0, allow
02/26-08:19:45.017007, 5, TCP, stream_tcp, 57, C2S, 192.168.17.20:34616, 192.168.17.30:80, 1:4000002:0, allow

There is an http.so file in the extras that could be giving you a problem (so leave that out).

Albert Lewis
ENGINEER.SOFTWARE ENGINEERING
SOURCEfire, Inc. now part of Cisco
Email: allewi () cisco com<mailto:allewi () cisco com>

From: Marcin Dulak <marcin.dulak () gmail com<mailto:marcin.dulak () gmail com>>
Date: Sunday, February 26, 2017 at 2:31 PM
To: allewi <allewi () cisco com<mailto:allewi () cisco com>>
Cc: 'snort-users' <snort-users () lists sourceforge net<mailto:snort-users () lists sourceforge net>>
Subject: Re: [Snort-users] snort3: problem with http_inspect

There is no nfqueue involved starting from this post http://seclists.org/snort/2017/q1/587
Getting rid of --plugin-path /usr/lib64/snort_extra makes the difference for me, but I need it due to 
http://seclists.org/snort/2017/q1/526
Can you confirm that by adding --plugin-path the problem exists?

Marcin

On Sun, Feb 26, 2017 at 7:17 PM, Al Lewis (allewi) <allewi () cisco com<mailto:allewi () cisco com>> wrote:
Try running it without nfq.

ALLEWI-M-8257:marcin-issue allewi$ ./bin/snort -c etc/snort/marcin.lua -r ~/Downloads/marcin-sent.pcap -Acsv -q
02/26-08:19:45.017007, 4, TCP, raw, 133, C2S, 192.168.17.20:34616<http://192.168.17.20:34616>, 
192.168.17.30:80<http://192.168.17.30:80>, 1:4000003:0, allow
02/26-08:19:45.017007, 5, TCP, stream_tcp, 57, C2S, 192.168.17.20:34616<http://192.168.17.20:34616>, 
192.168.17.30:80<http://192.168.17.30:80>, 1:3000002:0, allow
02/26-08:19:45.017007, 5, TCP, stream_tcp, 57, C2S, 192.168.17.20:34616<http://192.168.17.20:34616>, 
192.168.17.30:80<http://192.168.17.30:80>, 1:3000001:0, allow
02/26-08:19:45.017007, 5, TCP, stream_tcp, 57, C2S, 192.168.17.20:34616<http://192.168.17.20:34616>, 
192.168.17.30:80<http://192.168.17.30:80>, 1:4000002:0, allow



Albert Lewis
ENGINEER.SOFTWARE ENGINEERING
SOURCEfire, Inc. now part of Cisco
Email: allewi () cisco com<mailto:allewi () cisco com>

From: Marcin Dulak <marcin.dulak () gmail com<mailto:marcin.dulak () gmail com>>
Date: Sunday, February 26, 2017 at 9:25 AM
To: allewi <allewi () cisco com<mailto:allewi () cisco com>>
Cc: 'snort-users' <snort-users () lists sourceforge net<mailto:snort-users () lists sourceforge net>>
Subject: Re: [Snort-users] snort3: problem with http_inspect

The problem still there when replaying pcap, using build 227 
https://github.com/snortadmin/snort3/commit/89bae69d5cd980ae56ef0322b6ef7cca87a75cf2
I'm attaching the pcap, and the outputs of http_inspect/http_server:
# SNORT_LUA_PATH=/etc/snort LUA_PATH=/usr/include/snort/lua/?.lua snort --daq-dir /usr/lib64/daq -c 
/etc/snort/snort.lua --plugin-path /usr/lib64/snort_extra -R /etc/snort/rules/snort.rules -r test.pcap -A alert_fast -d

The rules are the same as before:
# cat /etc/snort/rules/snort.rules
alert tcp any any -> any 80 (msg:"test"; flow:to_server,established; http_uri; content:"/test"; sid:3000001;)
alert tcp any any -> any 80 (msg:"test"; http_uri; content:"/test"; sid:3000002;)
alert tcp any any -> any 80 (msg:"LOCAL http_method test for GET"; http_method; content: "GET"; sid:4000001;)
alert tcp any any -> any 80 (msg:"LOCAL http_method test for GET"; http_method; sid:4000002;)
alert tcp any any -> any 80 (msg:"LOCAL http_method test for GET"; content: "GET"; sid:4000003;)

To reproduce from a CentOS7 VM:

# cat /etc/yum.repos.d/copr-marcindulak-snort.repo
[copr-marcindulak-snort]
name=copr-marcindulak-snort
baseurl=https://copr-be.cloud.fedoraproject.org/results/marcindulak/snort/epel-$releasever-$basearch
enabled=1
gpgcheck=1
gpgkey=https://copr-be.cloud.fedoraproject.org/results/marcindulak/snort/pubkey.gpg

# yum -y install snort snort-extra

Marcin



On Sun, Feb 26, 2017 at 2:33 AM, Al Lewis (allewi) <allewi () cisco com<mailto:allewi () cisco com>> wrote:
I am using the default snort.lua (with http_inspect enabled). You really should have those comments removed for http 
inspection to work properly.

You can try running snort with the daq dump enabled to see the packets handled by snort.

Also check to see if the correct number of packets are in the exit stats (and not discarded).



Albert Lewis
ENGINEER.SOFTWARE ENGINEERING
SOURCEfire, Inc. now part of Cisco
Email: allewi () cisco com<mailto:allewi () cisco com>

From: Marcin Dulak <marcin.dulak () gmail com<mailto:marcin.dulak () gmail com>>
Date: Saturday, February 25, 2017 at 6:19 PM
To: allewi <allewi () cisco com<mailto:allewi () cisco com>>
Cc: 'snort-users' <snort-users () lists sourceforge net<mailto:snort-users () lists sourceforge net>>
Subject: Re: [Snort-users] snort3: problem with http_inspect



On Sat, Feb 25, 2017 at 11:24 PM, Al Lewis (allewi) <allewi () cisco com<mailto:allewi () cisco com>> wrote:
Hello,

        I
think you need to uncomment http_inspect “remove the dashes from in front of it”

the behavior of http_inspect I described was without any dashes, with the default snort.lua from github.
I have tested whether the lua comment "--" makes any difference and it does not - I mean dashes are treated as a 
comment.

-- http_inspect = { }
http_server = { }

Are you using the default lua files from github? Or maybe the few last commits since 
https://github.com/snortadmin/snort3/commit/a9f9bd38ced24da8196746074ef60a73d3bf0438 could have changed something?
Or maybe related to hyperscan, which I'm not using?

Marcin


It alerts for me.


ALLEWI-M-8257:snort3 allewi$ ./bin/snort -c etc/snort/marcin.lua -r /tmp/TEST.pcap -Acmg -k none -q
02/25-16:54:57.819915 [**] [1:3000001:0] "test" [**] [Priority: 0] {TCP} 
192.168.1.128:53687<http://192.168.1.128:53687> -> 74.125.196.99:80<http://74.125.196.99:80>
- - - stream_tcp[58]- - - - - - - - - - - - - - - - - - - - - - - - -
48 6F 73 74 3A 20 77 77 77 2E 67 6F 6F 67 6C 65  Host: www.google
2E 63 6F 6D 0D 0A 55 73 65 72 2D 41 67 65 6E 74  .com..User-Agent
3A 20 63 75 72 6C 2F 37 2E 34 33 2E 30 0D 0A 41  : curl/7.43.0.<http://7.43.0.>.A
63 63 65 70 74 3A 20 2A 2F 2A                    ccept: */*
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

ALLEWI-M-8257:snort3 allewi$ cat etc/snort/marcin.lua | grep alert
        alert tcp any any -> any 80 (msg:"test"; flow:to_server,established;http_uri; content:"/test"; sid:3000001;)
ALLEWI-M-8257:snort3 allewi$






Albert Lewis
ENGINEER.SOFTWARE ENGINEERING
SOURCEfire, Inc. now part of Cisco
Email: allewi () cisco com<mailto:allewi () cisco com>








On 2/25/17, 2:05 PM, "Marcin Dulak" <marcin.dulak () gmail com<mailto:marcin.dulak () gmail com>> wrote:

> Hi,
>
> I have a problem with http_inspect,
> https://github.com/snortadmin/snort3/commit/a9f9bd38ced24da8196746074ef60a73d3bf0438
>
> I make an HTTP request against the machine running snort/nfqueue:
>
> # curl -s -m 1 http://192.168.17.30/test
>
> and expect my sid:3000001 (see below) to be triggered, but only sid:4000003
> is triggered instead.
> My question is what am I missing to trigger sid:3000001 with the new
> http_inspect?
>
> Now, when in /etc/snort/snort.lua I use
> -- http_inspect = { }
> http_server = { }
>
> then all but sid:4000001 are triggered:
>
> # u2spewfoo /var/log/snort/unified2.log.1488047835 | grep "sig id"
>    sig id: 4000003    gen id: 1    revision: 0     classification: 0
>    sig id: 3000002    gen id: 1    revision: 0     classification: 0
>    sig id: 3000001    gen id: 1    revision: 0     classification: 0
>    sig id: 4000002    gen id: 1    revision: 0     classification: 0
>
> I see the unified2 log contains also (ExtraDataHdr) (ExtraData)
> and only two events get parsed by py-idstools, which I normally use with
> snort2:
>
> # idstools-u2json /vagrant/unified2.log.1488047835 | grep signature
> WARNING: No alert message map entries loaded.
> WARNING: No classifications loaded.
> ERROR: Unknown record type: 3
> {"event": {"dport-icode": 80, "pad2": 0, "event-second": 1488047842,
> "sensor-id": 0, "event-id": 1, "classification-id": 0, "sport-itype":
> 40062, "generator-id": 1, "signature-revision": 0, "mpls-label": 0,
> "event-microsecond": 283661, "protocol": 6, "destination-ip":
> "192.168.17.30", "blocked": 0, "signature-id": 4000003, "priority": 0,
> "vlan-id": 0, "impact-flag": 0, "impact": 0, "source-ip": "192.168.17.20"}}
> {"event": {"dport-icode": 80, "pad2": 0, "event-second": 1488047842,
> "sensor-id": 0, "event-id": 2, "classification-id": 0, "sport-itype":
> 40062, "generator-id": 1, "signature-revision": 0, "mpls-label": 0,
> "event-microsecond": 283661, "protocol": 255, "destination-ip":
> "192.168.17.30", "blocked": 0, "signature-id": 3000002, "priority": 0,
> "vlan-id": 0, "impact-flag": 0, "impact": 0, "source-ip": "192.168.17.20"}}
>
> Snort running as:
>
> # xargs -0 < /proc/`pidof snort`/cmdline
> /usr/sbin/snort -d -Q --daq-dir /usr/lib64/daq --daq nfq -l /var/log/snort
> -c /etc/snort/snort.lua -A unified2 -v -X --plugin-path
> /usr/lib64/snort_extra -k none
>
> # iptables-save
> *filter
> :INPUT ACCEPT [5428:45165731]
> :FORWARD ACCEPT [0:0]
> :OUTPUT ACCEPT [4796:239048]
> -A INPUT -i enp0s8 -j NFQUEUE --queue-num 0 --queue-bypass
> -A OUTPUT -o enp0s8 -j NFQUEUE --queue-num 0 --queue-bypass
> COMMIT
>
> The only difference compared to the github's lua files is in
> /etc/snort/snort_defaults.lua
>
> # diff snort3/lua/snort_defaults.lua /etc/snort/snort_defaults.lua
> 32a33,35
> > RULE_PATH = conf_dir .. '/rules'
> > ips = { include = RULE_PATH .. '/snort.rules' }
>
> and the rules as follows:
>
> # cat /etc/snort/rules/snort.rules
> alert tcp any any -> any 80 (msg:"test"; flow:to_server,established;
> http_uri; content:"/test"; sid:3000001;)
> alert tcp any any -> any 80 (msg:"test"; http_uri; content:"/test";
> sid:3000002;)
> alert tcp any any -> any 80 (msg:"LOCAL http_method test for GET";
> http_method; content: "GET"; sid:4000001;)
> alert tcp any any -> any 80 (msg:"LOCAL http_method test for GET";
> http_method; sid:4000002;)
> alert tcp any any -> any 80 (msg:"LOCAL http_method test for GET"; content:
> "GET"; sid:4000003;)
>
>
> Marcin
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net<mailto:Snort-users () lists sourceforge net>
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>
> Please visit http://blog.snort.org to stay current on all the latest Snort news!



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!