Re: http_inspect missing requests

[Russ <rucombs () cisco com>]

The http_inspect preprocessor has evolved over the years to become more 
stateful but retains some stateless processing which your new pcaps are 
exercising since they lack a full TCP session with 3-way handshake.  
Processing the bald data segments can lead to bogus results along with 
diminished performance.

Consider the pcap with 10 fully overlapping segments.  Snort processed 
them all.  Within the context of a normal session, only one would be 
processed depending upon policy because only one would be delivered to 
the receiving TCP user.  In IDS mode Snort will handle the overlaps 
according to configured policy whereas in IPS mode Snort will ensure 
first wins and normalize subsequent overlaps to match.  So, normal 
traffic with a proper session will be processed more efficiently and 
more accurately.

If you are curious, try crafting a full session for these two cases and 
see how it goes.  If you are extra curious, try out Snort++ instead 
which has a completely new http_inspect.

On 2/8/17 6:39 AM, Felix Erlacher wrote:
> Thanks for the help.
> All GET requests where processed in inline mode like you proposed. Is
> this because in IDS mode Snort works in post-ack inspection mode and in
> inline (IPS) mode it does pre-ack inspection?
> I couldn't find any information about this in the Snort manual.
>
> But there are still some questions regarding this trace.
> You say that if packets are not ACKed, Snort will not look at them (if
> not in IPS mode).
> But if I put the same TCP payload in one segment (10GETonePanon.pcap)
> and feed it to Snort, the http_inspect stats show me 10 GET requests.
> But according to your last mail it shouldn't because the segment is not
> ACKed.
> (Again, I used the standard snort.conf from 2.9.9.0 in IDS mode with the
> -k none switch)
>
> The same holds if I put every GET request in an individual packet,
> resulting in 10 TCP segments (10indivGETanon.pcap). http_inspect tells
> me it processed 10 GET requests altough none of the 10 packets was
> ACKed. (They even have all the same SEQ numbers.)
>
> There is one difference betwee the two traces, though. The rule with sid
> 2013504 from the Emerging Threats ruleset looks for
> content:"APT-HTTP|2F|" in the http_header.
> It fires 5 alerts for the 10GETonePanon.pcap trace but 10 alerts for the
> 10indivGETanon.pcap trace. The payload can be found 10 times in both traces.
>
> It would be great if someone could give me some insights on this.
>
> greets
>
> felix
>
>
> On 03/02/17 23:06, Russ wrote:
> > The final 3 GET requests were not acknowledged by the TCP server and so
> > weren't processed.  If you run in IPS mode you will see them get them
> > processed.  To enable IPS mode, make sure you have
> >
> >      preprocessor normalize_tcp: ips
> >
> > in your conf and add these args to your command line:
> >
> >      --daq dump --daq-var load-mode=read-file -Q
> >
> > The dump DAQ allows you to test inline mode with pcaps (it will create a
> > new pcap with only the packets allowed to pass); -Q enables inline mode;
> > and normalize_tcp: ips enables stream normalization.
> >
> > On 2/3/17 1:27 PM, Felix Erlacher wrote:
> > > Hi all,
> > >
> > > I have a pcap trace containing HTTP traffic. I began to wonder because
> > > Snort did not trigger all alerts I was expecting. So I extracted the TCP
> > > stream in question and looked at it more closely. My impression is that
> > > for some reason the HTTP preprocessor is not parsing all GET requests.
> > > If I load this trace in Wireshark, than "follow TCP stream", it shows me
> > > 10 GET requests.
> > > If I use ngrep to manually inspect the trace, I count 10 GET requests as
> > > well.
> > >
> > > But the HTTP Inspect preprocessor of Snort tells me it found only 7 GET
> > > requests?!
> > > What could possibly be the problem?
> > >
> > > Some peculiarities of the trace:
> > > Heavy usage of HTTP/1.1 pipelining
> > > While Wireshark and the Snort DAQ tell me they processed 13 packets,
> > > HTTP inspect tells me it processed 17 packets.
> > > This trace contains checksum errors and a tcp RST in the last packet.
> > >
> > > I am using Snort 2.9.9.0 with snort.conf from tarball and "-k none" switch.
> > >
> > > I would be happy to share the trace, but for privacy reasons I don't
> > > want to do that on the list. In case someone wants to take a look, just
> > > drop me a mail.
> > >
> > > thanks and greetings
> > >
> > >
> > > ------------------------------------------------------------------------------
> > > Check out the vibrant tech community on one of the world's most
> > > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > >
> > >
> > > _______________________________________________
> > > Snort-users mailing list
> > > Snort-users () lists sourceforge net
> > > Go to this URL to change user options or unsubscribe:
> > > https://lists.sourceforge.net/lists/listinfo/snort-users
> > > Snort-users list archive:
> > > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
> > >
> > > Please visit http://blog.snort.org to stay current on all the latest Snort news!


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!