Re: HTTP Reassembly issue PAF enabled

[Parmendra Pratap <parmendra.pratap () yahoo com>]

Exact command I am using is:
snort --daq dump --daq-var load-mode=read-file -Q -i eth0 -c etc/snort.conf 
Fails to start with following error
Can't initialize DAQ dump (-1) - eth0: No such file or directory

No typo issue as such, but its perhaps just that I am trying to run it from a device instead of a pcap file.

Following works for me as desired:

snort --daq dump --daq-var load-mode=passive -Q -i eth0 -c etc/snort.conf


Thanks file=/dev/null takes care of the dump file and disk saturation.

I do get why normalization in general makes more sense for inline mode, but putting reassembly "first" seems to be 
something just not exclusive to inline mode.
Would be great if there was a config hook to enable/disable this in NIDS mode.
I guess its more of a matter of opinion :)

Thanks for the help. 
The promptness here is impressive and motivates me more to go the snort route for my current IDS and n/w traffic 
analysis needs.

Thanks
Parmendra


________________________________
 From: Russ Combs <rcombs () sourcefire com>
To: Parmendra Pratap <parmendra.pratap () yahoo com> 
Cc: "snort-devel () lists sourceforge net" <snort-devel () lists sourceforge net> 
Sent: Monday, 8 April 2013 6:33 PM
Subject: Re: [Snort-devel] HTTP Reassembly issue PAF enabled
 
On Mon, Apr 8, 2013 at 12:51 PM, Parmendra Pratap
<parmendra.pratap () yahoo com> wrote:
> Instead of --daq dump --daq-var load-mode=read-file , I used --daq dump
> --daq-var load-mode=passive, as load_mode=read_file was not starting and
> looking for a file path.

Maybe just a typo, but that should be "load-mode=read-file" ( '-' not '_' ).
> I can see its functionally working for sure as far as TCP headers and HTTP
> PDU reassembly is concerned.
>
> There are couple of issues though - it creates a dump file, namely
> inline-pcap.out file which keeps growing until I run out of disk space.
> Isn't there a way to turn tcp normalization ON on without needing to dump
> pcap to file system.
> Performance seems to be OK but would defeinitely be desirable from my end to
> be able to do TCP normalization without needing to dump to the filesystem.
>
> Is there a reason behind deactivating TCP Normalization in IDS mode?
Yes - normalizations require inline operation.  No point changing a
packet if isn't being forwarded.  Also, stream5 tries to reassemble
the way the receiving host does.  This won't be the case if not
inline.

You can try something like --daq-var file=/dev/null if you don't want
to write the file.

> Regards
> Parmendra
>
>
> From: Russ Combs <rcombs () sourcefire com>
> To: Parmendra Pratap <parmendra.pratap () yahoo com>
> Cc: "snort-devel () lists sourceforge net" <snort-devel () lists sourceforge net>
> Sent: Monday, 8 April 2013 3:36 PM
>
> Subject: Re: [Snort-devel] HTTP Reassembly issue PAF enabled
>
> On Fri, Apr 5, 2013 at 7:14 PM, Parmendra Pratap
> <parmendra.pratap () yahoo com> wrote:
> > Brilliant - Thanks Russ normalization on tcp sorts this out.
> > Just a a small note - I used load_mode=passive.
>
> Not clear what you did but glad it works for you.
>
> > So if I go it right with normalization on , reverse Ack is not awaited for
> > in S5 prior to flushing , rest is same and PAF still works as expected to
> > for supported protocols/preprocessors ?
>
> Yes
> > Also , is there at all a flip side to running snort in "forced" inline
> > mode
> > on top pcap like this, performance wise may be?
>
> --daq dump --daq-var load-mode=read-file will be slower than straight
> readback because it also writes to file all packets that weren't
> blocked as well as any injected packets.  Hopefully performance isn't
> the major concern in readback mode.
>
> > Thanks
> > Parmendra
> >
> > ________________________________
> > From: Russ Combs <rcombs () sourcefire com>
> > To: Parmendra Pratap <parmendra.pratap () yahoo com>
> > Cc: "snort-devel () lists sourceforge net"
> > <snort-devel () lists sourceforge net>
> > Sent: Friday, 5 April 2013 5:15 PM
> > Subject: Re: [Snort-devel] HTTP Reassembly issue PAF enabled
> >
> > On Fri, Apr 5, 2013 at 11:56 AM, Parmendra Pratap
> > <parmendra.pratap () yahoo com> wrote:
> > > Hi Hui
> > >
> > > Thanks again for a quick response.
> > >
> > > Have tried -Q flag , but since I am using Pcap DAQ it fails to start with
> > > -Q
> > > set.
> > > However it does start wiith --test-inline mode , which I assume works no
> > > different from -Q except that drops are not enabled.
> >
> > Actually, normalizations are not enabled in that case.  You should see
> > something like this in your startup output:
> >
> > WARNING: tcp normalizations disabled because not inline.
> >
> > Try running with:
> >
> > --daq dump --daq-var load-mode=read-file -Q
> >
> > > My snort.conf does have preprocessor normalize_tcp: ips have set.
> > >
> > > But even with the above set up I can still replicate the same issue
> > > related
> > > to incorrect tcp flags.
> > > As I said before , I think it seems like a matter of setting the tcp
> > > header
> > > of the last packet that completes the PDU inside Packet->tcph->th_flags
> > > in
> > > the s5 preprocessor when doing a PAF based flushing.
> > > I can see that the direction i.e. sourceIP/port and destIP/port being
> > > reversed for the same reason in the Packet struct when doing a reassembly
> > > based flushing from s5.
> > >
> > > Thanks
> > > Parmendra
> > >
> > >
> > > __________________________________________________________________________
> > >
> > > Hi Parmendra,
> > >
> > > To be clear, you must use IPS mode to get what you want, so you need to
> > > 1) use -Q  when you run snort
> > > 2) Enable Normalization for TCP:
> > >
> > > preprocessor normalize_tcp: ips
> > >
> > > Best,
> > > Hui.
> > > On 04/04/2013 08:25 AM, Parmendra Pratap wrote:
> > > > Hi Hui
> > > > Thanks for a quick reply.
> > > > I tried the use case with Snort 2.9.4.5.
> > > > Does not make a difference.
> > > > Issue is still replicable with the steps outlined in my root email.
> > > > This is what I think is going on , based on few tests and source
> > > > lookups <excuse my newbieness if it reflects anywhere below :) >-
> > > > Stream5 reassembly does not tag a packet as complete PDU until it
> > > > recieves subsequent ack gainst the packet no matter wheter or not the
> > > > packet actually holds complete PDU  (in this case HTTP) or not.
> > > > With PAF enabled this prevents the URI Bufs from being created and
> > > > inspected in HTTP inspect module until the next packet arrives (ie ack
> > > > against the original packet that contained the HTTP req).
> > > > When the HTTP inspect URI Bufs based match fires, with PAF ON,  its
> > > > always(mostly?) when the ack on reverse direction is received.
> > > > The spo_alert* modules simply uses the header data provided in the
> > > > Packet->tcph which holds the header from the current packet ie ack
> > > > packet from the server ... and hence the incorrect TCP header display
> > > > with PAF on.
> > > > There is no way curently to get the correct TCP headers unless Stream
> > > > 5 is queried to give the original raw packet <spo_log_tcp_dump.c does
> > > > that>.
> > > > Realworld issue arising from the above is incorrect TCP header data in
> > > > alerts. TCP dumps are OK though for reason mentioned above.
> > > > There seems multiple ways to get around this:
> > > > Generate TCP dump on all alerts and assume the alerts will have
> > > > incorrect TCP headers
> > > > Write an alert output plugin that inspects the raw packet for correct
> > > > TCP headers etc
> > > > Add more metadata to Packet struct which can provide the correct TCP
> > > > headers at least for the last packet that completed the PDU in the
> > > > alert output plugins.
> > > > Last option looks the most organic and least sub optimal one to me.
> > > > Given my little experience with snort so far , I wont be surprised if
> > > > any of the above stated flow is incorrect.
> > > > I will more than appreciate if someone can correct me above and
> > > > enlighten me more about the internals :) .
> > > > Thanks
> > > > Parmendra
> > >
> > >
> > >
> > >
> > > ------------------------------------------------------------------------------
> > > Minimize network downtime and maximize team effectiveness.
> > > Reduce network management and security costs.Learn how to hire
> > > the most talented Cisco Certified professionals. Visit the
> > > Employer Resources Portal
> > > http://www.cisco.com/web/learning/employer_resources/index.html
> > > _______________________________________________
> > > Snort-devel mailing list
> > > Snort-devel () lists sourceforge net
> > > https://lists.sourceforge.net/lists/listinfo/snort-devel
> > > Archive:
> > > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel
> > >
> > > Please visit http://blog.snort.org/ for the latest news about Snort!
------------------------------------------------------------------------------
Minimize network downtime and maximize team effectiveness.
Reduce network management and security costs.Learn how to hire 
the most talented Cisco Certified professionals. Visit the 
Employer Resources Portal
http://www.cisco.com/web/learning/employer_resources/index.html

_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about Snort!