Re: preprocessors

[Matt Olney <molney () sourcefire com>]

OK, I failed the retard check, and Finchy called me on it.  I'll plead a
combination of sick and having a 2 and 6 year old having a royal rumble in
the computer room.

Anyways, real quick revisit on frag3 and stream:

Frag3 handles layer 3 (and specifically IP) fragmentation.  It reassembles
packets that have been fragmented as a result of MTU differences, or
intentionally by an attacker as an evasion technique.
Stream5 handles a few things.  Primarily it is responsible
for reassembling TCP segments (again, happens in normal traffic, but also
happens as an evasion technique).  It also sets up some structures on the
back end that allow the rules to query the flow direction and whether that
flow is going to the server or the client.  The rule option flow: to_server,
established; is possible in TCP rules because of the stream5 engine.  The
stream5 engine also performs a similar check on UDP traffic (no reassembly
necessary here) so we can do flow: to_server and flow: to_client on UDP
rules.

Sorry for the screw up, and thanks Joel for the gentle email :)

Matt

On Wed, Dec 16, 2009 at 8:13 AM, Matt Olney <molney () sourcefire com> wrote:

> Hey Jonas,
>
> That's a pretty quality set of questions.  Let me take a crack, and maybe
> some folks can chime in as well.
>
> 1)  The preprocessors work in the order you have them in the config file.
>  So first the frag3 engine cleans up layer 2 fragmentation.  Then the stream
> engine handles the reassembly of IP segmentation.  Then (for example) the
> http_inspect engine applies some intelligence to the data and sorts it into
> buffers that we can specifically look at in the detection engine.  This way
> we can write rules that are faster and more accurate.
>
> 2)  In the SFPortscan preprocessor, detection was added because we wanted
> to alert on the overall behavior of a host.  The detection engine is built
> to be packet and stream centric and it's tricky (not completely impossible
> in .SO rules, but tricky) to hold data across streams.  The SFPortscan is
> more efficient in doing this and provides a unified place for this sort of
> detection.  Detection is built into other preprocessors because, from a
> speed point of view, it is quicker to handle a set of data and look at
> several different potential issues, rather than trying to write a set of
> rules that provide the level of detection and and speed we need.  For
> example, the http_inspect engine already breaks out the URI into a separate
> buffer and normalizes the data.  Why not look right there if that URI is
> suspicious, either because there appears to be directory traversal or URI
> encoding (both of which are corrected in the normalization process, although
> the raw data is still available to look at if we needed to) or if it is too
> long.  It is much easier to that detection when you are parsing the data.
>
> The decode, which is a sort of preprocessor, but one I kind of set aside in
> my mind as 'different' now contains a lot of the functionality that used to
> be in rules.  For example, weird tcp flags set ups are now handled here,
> because you are handling the data anyways and to do the same test in the
> detection engine is much less efficient, because of how the engine and rule
> language work.
>
> Hope that answers your questions.
>
> Matt Olney
> Research Engineer
> Sourcefire VRT
>
>
> On Wed, Dec 16, 2009 at 7:56 AM, Jonas Pfoh <pfoh () sec in tum de> wrote:
>
> > Hi,
> >
> > I have a two questions to using preprocessors.
> >
> > 1. Do I understand correctly that preprocessors such as frag3 do some
> > preprocessing (in the case of frag3, assemble packets), then send them
> > along to the detection engine to be analyzed?  Clearly it makes sense
> > that they do as they are called "preprocessors", but it brings me to my
> > next question...
> >
> > 2. Preprocessors like sfPortscan, seem to do less preprocessing and more
> >  alerting...shouldn't this be the job of the detection engine?  Is it
> > done in a preprocessor, because state is needed?  When an alert is
> > triggered by the preprocessor, is/are the packet(s) still sent to the
> > detection engine?
> >
> > Thanks for you help.
> >
> > -Jonas
> >
> >
> > ------------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Verizon Developer Community
> > Take advantage of Verizon's best-in-class app development support
> > A streamlined, 14 day to market process makes app distribution fast and
> > easy
> > Join now and get one step closer to millions of Verizon customers
> > http://p.sf.net/sfu/verizon-dev2dev
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users () lists sourceforge net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users<https://lists.sourceforge.net/lists/listinfo/snort-usersSnort-users>list archive:
> > http://www.geocrawler.com/redir-sf.php3?list=snort-users
------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users