Re: XFF/ExtraData not always logged for drop rules

[Mike Cox <mike.cox52 () gmail com>]

Thanks Carter.  Is my analysis correct on how XFF logging works and the
lack of logging in the described scenarios?

Thanks!

-Mike Cox

On Thu, Jun 25, 2015 at 2:08 PM, Carter Waxman (cwaxman) <cwaxman () cisco com>
wrote:

>  Hi Mike,
>
>  Thanks for the detailed analysis. We will be tracking this internally.
>
>  -Carter
>
>   From: Mike Cox <mike.cox52 () gmail com>
> Date: Thursday, June 25, 2015 at 2:00 PM
> To: "snort-devel () lists sourceforge net" <snort-devel () lists sourceforge net
> >
> Subject: Re: [Snort-devel] XFF/ExtraData not always logged for drop rules
>
>     I did some quick tests with Snort 2.9.7.3/Stream6 and saw the same
> behavior described above.
>
>  Any insights, agreements, or disagreements?
>
>  Testing scenario 1 is easy enough.  For scenario 2, I've attached a pcap
> with a simple HTTP request (with XFF header) and response that both require
> reasembly.
>
>  These rules can be used to test (only enable one at a time):
>
>  drop tcp any any -> any $HTTP_PORTS (msg:"Client request with XFF -
> reassembly required"; flow:established,to_server; content:"GET";
> http_method; content:"requestmatch"; sid:36332;)
>
> drop tcp any $HTTP_PORTS -> any any (msg:"Server response XFF test -
> reassembly required"; flow:established,to_client; content:"Server";
> http_header; file_data; content:"ninja"; sid:26221;)
>
>  If 'normalize: tcp' is enabled (flush policy is *-IPS/pre-ACK), the the
> rules will trigger but the ExtraData with the XFF info is not logged.  If
> you change the rules to 'alert' or if you disable normalize so the sensor
> is in post-ACK mode, the ExtraData/XFF info is logged.  You'll want to run
> in inline mode (the dump DAQ is great for testing!) and of course you need
> to be configured to output unified2.
>
>  Thanks!
>
>  -Mike Cox
>
> On Wed, Jun 24, 2015 at 9:49 AM, Mike Cox <mike.cox52 () gmail com> wrote:
>
> >  VRT and Snort Devs,
> >
> >
> > The X-Forwared-For ("XFF") and/or True-Client-IP data is stored in what is called "ExtraData" and written to the 
> > unified2 file.  ExtraData is often not written to the unified2 file at the same time as other alert data and can 
> > depend on stream flushing as well as alert flushing.
> >
> > Looking at Stream5, there are a number of cases where the XFF ExtraData is not logged on 'drop' rules even though it 
> > is available in the data stream.  From what I can tell, the ExtraData gets written by purge_alerts() which gets 
> > called by purge_to_seq() which gets called by purge_flushed_ackd() (sometimes these functions get called when 
> > dealing with TIME_WAIT timer stuff but let's ignore those cases for now).  However, purge_flushed_ackd() only purges 
> > flushed and acked bytes (as I understand it, bytes that are flushed *and* ackd).
> >
> > So in these situations, when 'drop' rules trigger, the ExtraData is not written:
> >
> > 1) A single packet triggers the drop rule and it is inspected as a packet and not part of a (reassembled) stream.
> >   - The Stream5 preprocessor doesn't get involved enough to do the appropriate flushing/purging required to write 
> > the ExtraData.
> >
> > 2) A reassembled stream "packet" triggers the drop rule *and* the normalize_tcp preprocessor is enabled (i.e. 
> > 'normalize_tcp: ips' which is going to be the Protocol-IPS or Footprint-IPS flush policy depending on PAF).
> >   - Snort is in pre-ACK mode and so it doesn't wait on the ACK to flush the data to detection.  Since the flushing 
> > happens before the ACK is received (and the ACK isn't processed anyway since the stream is blocked by the block 
> > rule), the ExtraData never gets written.
> >
> > I can understand and somewhat accept why the ExtraData isn't written for scenario 1 although this happens when the 
> > HTTP Inspect preprocessor is already engaged so it seems feasible to log the ExtraData/XFF.  Can this be done?
> >
> > For scenario 2, can I make a feature request that the ExtraData gets logged appropriately in this case?  I'm 
> > guessing that people who run Snort inline also have normalize and PAF enabled and it makes sense to me that 'drop' 
> > rules would still write ExtraData, especially since Stream5 is fully involved.
> > Once drop rules fire, the stream gets blocked (assuming the DAQ supports this) so it makes sense to go ahead and 
> > compile/write out the ExtraData since nothing else on that stream is going to get fully processed.
> >
> > I haven't looked much at Stream6 although it looks like most of the code from Stream5 so I'm not sure why the 
> > version number change.
> >
> > Thanks!
> >
> > -Mike Cox
> >
> > P.S. setting 'flush_on_alert' for Stream5 doesn't seem to have any affect on these scenarios.
------------------------------------------------------------------------------
Monitor 25 network devices or servers for free with OpManager!
OpManager is web-based network management software that monitors 
network devices and physical & virtual servers, alerts via email & sms 
for fault. Monitor 25 devices for free with no restriction. Download now
http://ad.doubleclick.net/ddm/clk/292181274;119417398;o

_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about Snort!