Re: preprocessors

[Matt Olney <molney () sourcefire com>]

Hey Jonas,

That's a pretty quality set of questions.  Let me take a crack, and maybe
some folks can chime in as well.

1)  The preprocessors work in the order you have them in the config file.
 So first the frag3 engine cleans up layer 2 fragmentation.  Then the stream
engine handles the reassembly of IP segmentation.  Then (for example) the
http_inspect engine applies some intelligence to the data and sorts it into
buffers that we can specifically look at in the detection engine.  This way
we can write rules that are faster and more accurate.

2)  In the SFPortscan preprocessor, detection was added because we wanted to
alert on the overall behavior of a host.  The detection engine is built to
be packet and stream centric and it's tricky (not completely impossible in
.SO rules, but tricky) to hold data across streams.  The SFPortscan is more
efficient in doing this and provides a unified place for this sort of
detection.  Detection is built into other preprocessors because, from a
speed point of view, it is quicker to handle a set of data and look at
several different potential issues, rather than trying to write a set of
rules that provide the level of detection and and speed we need.  For
example, the http_inspect engine already breaks out the URI into a separate
buffer and normalizes the data.  Why not look right there if that URI is
suspicious, either because there appears to be directory traversal or URI
encoding (both of which are corrected in the normalization process, although
the raw data is still available to look at if we needed to) or if it is too
long.  It is much easier to that detection when you are parsing the data.

The decode, which is a sort of preprocessor, but one I kind of set aside in
my mind as 'different' now contains a lot of the functionality that used to
be in rules.  For example, weird tcp flags set ups are now handled here,
because you are handling the data anyways and to do the same test in the
detection engine is much less efficient, because of how the engine and rule
language work.

Hope that answers your questions.

Matt Olney
Research Engineer
Sourcefire VRT

On Wed, Dec 16, 2009 at 7:56 AM, Jonas Pfoh <pfoh () sec in tum de> wrote:

> Hi,
>
> I have a two questions to using preprocessors.
>
> 1. Do I understand correctly that preprocessors such as frag3 do some
> preprocessing (in the case of frag3, assemble packets), then send them
> along to the detection engine to be analyzed?  Clearly it makes sense
> that they do as they are called "preprocessors", but it brings me to my
> next question...
>
> 2. Preprocessors like sfPortscan, seem to do less preprocessing and more
>  alerting...shouldn't this be the job of the detection engine?  Is it
> done in a preprocessor, because state is needed?  When an alert is
> triggered by the preprocessor, is/are the packet(s) still sent to the
> detection engine?
>
> Thanks for you help.
>
> -Jonas
>
>
> ------------------------------------------------------------------------------
> This SF.Net email is sponsored by the Verizon Developer Community
> Take advantage of Verizon's best-in-class app development support
> A streamlined, 14 day to market process makes app distribution fast and
> easy
> Join now and get one step closer to millions of Verizon customers
> http://p.sf.net/sfu/verizon-dev2dev
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users