Re: http_header issues, Snort 2.8.5.3

[Will Metcalf <william.metcalf () gmail com>]

I guess it should be noted for any interested parties,  if you are
writing a sig for suricata using snort rule syntax the cookie will be
included as part of the host_header buffer.  We will document this and
other deviations from snort behavior in our release notes or maybe a
separate README included in doc/ .

I continue to send this stuff as I feel that it in every ones best
interest if intended behavior is well documented. It is important to
OISF as we continue to add support for snort rule language but I feel
it is equally as important for snort users and sig writers as well.  I
have gotten sort of side-tracked with documenting the snort
content/modifier behavior but I have some help now so we should wrap
it up pretty soon.

We will make our results available via spreadsheet. We will also make
available  the rules and pcaps used to test so that people can test on
try it on their own if they wish. Maybe the community can keep the
spreadsheet updated between snort releases or something, to alleviate
some of the confusion about how matches/modifiers interact with each
other.  I've even seen older revisions of VRT rules where there was
incorrect use of http_* modifiers appended to multiple content matches
and used in combo with distance/within which isn't supported BTW.  Not
trying to pick on VRT, just trying to point out that it can be really
confusing.

Regards,

Will

On Thu, Apr 1, 2010 at 10:33 AM, Steven Sturges
<steve.sturges () sourcefire com> wrote:
> That separation is all about a performance optimization for the
> fast pattern matcher.  If you also notice in the documentation,
> it mentions that the cookie cannot be used in conjunction with
> fast_pattern.  The cookie is not searched with fast pattern contents.
>
> If you have a 2000 byte HTTP Cookie, there is little use searching
> all of that data for content with the fast pattern matcher, which
> is used to select rules for further inspection.
>
> In 2.8.6, there is a configuration option that allows you to disable
> cookie extract, thereby keeping HTTP cookies with the rest of the
> HTTP headers.
>
> Typically, you won't find something in a cookie header field that
> you would also want to find in another header -- Host, Content-Type,
> etc.  When the extraction of various pieces of the HTTP requests was
> added in 2.8.3 -- a good 2+ years ago -- the development team
> discussed with VRT and we came to the conclusion that for best
> performance and flexibility, it was easiest to separate the two.
>
> There is nothing preventing a rule writer from using BOTH http_cookie
> and http_header as modifiers to the same content, aside from other
> limitations on http_cookie.
>
> Cheers.
> -steve
>
> Mike Cox wrote:
> > Agreed, I'm shocked that the http_header buffer doesn't include the
> > Cookie header.  It doesn't make sense.  According to the manual, "The
> > http header keyword is a content modifier that restricts the search to
> > the extracted Header fields of an HTTP client
> > request." (as an aside, note the incorrect use of 'an' instead of 'a')
> >  Why is the Cookie header a second class citizen in the HTTP headers
> > world?  I understand having a separate http_cookie buffer but it
> > doesn't mean Cookies are not headers anymore....
> >
> > -Mike Cox
> >
> > On 4/1/10, evilghost () packetmail net <evilghost () packetmail net> wrote:
> > > Thanks Will for the speedy response, I apologize for not have read your
> > > response to the list earlier.  I agree with you regarding this and it's
> > > counter-intuitive to have the Cookie removed from the http_header buffer.
> > >
> > > -evilghost
> > >
> > > Will Metcalf wrote:
> > > > That because the cookie isn't included in the normalized header
> > > > buffer, you can only get to via http_cookie modifer.  I know it
> > > > doesn't make any sense to me either.  I sent the following e-mail to
> > > > snort-devel on 3/17.
> > > >
> > > > "This is just my 2 cents, but I don't think the following behavior
> > > > makes sense.  I think that even though you are providing http_cookie
> > > > as a separate buffer to match on it should still be included in the
> > > > http_header buffer, well because it is part of the headers.
> > > >
> > > > You can still match using the raw buffer but then you have to add
> > > > additional checks to try and differentiate between the headers and the
> > > > body which is why I'm guessing these modifiers were created in the
> > > > first place.  I realize that in most cases header order doesn't matter
> > > > but there may be instances where you can fingerprint a piece of
> > > > automated code (read malware) using a rule similar to sid 69 below.
> > > >
> > > > Regards,
> > > >
> > > > Will"
> > > >
> > > > #this matches but I loose the performance/accuracy benefit of only
> > > > matching within the buffer containing http_headers.
> > > > alert tcp any any -> any any (msg:"http_cookie + ";
> > > > content:"Cookie|3A|
> > > > e6504ae48c99f09df7f58996aacbb6b0=120e494ce857d6ceeef89f9678d4d703|0d
> > > > 0a|Content-Type|3A| application"; classtype:bad-unknown; sid:68;
> > > > rev:1;)
> > > >
> > > > #this fails to match as the cookie is not part of the http_header
> > > > buffer but is part of the real http headers.
> > > > alert tcp any any -> any any (msg:"http_cookie + ";
> > > > content:"Cookie|3A|
> > > > e6504ae48c99f09df7f58996aacbb6b0=120e494ce857d6ceeef89f9678d4d703|0d
> > > > 0a|Content-Type|3A| application"; http_header; classtype:bad-unknown;
> > > > sid:69; rev:1;)
> > > >
> > > >
> > > >
> > > > On Thu, Apr 1, 2010 at 9:22 AM, evilghost () packetmail net
> > > > <evilghost () packetmail net> wrote:
> > > >
> > > > > Hello, I am running Snort 2.8.5.3 and it appears that either
> > > > > http_header; is not working correctly, does not work with a relative
> > > > > keyword, or I do not understand http_header; correctly.  I am attempting
> > > > > to constrain a content match to the http_header for performance reasons.
> > > > >
> > > > > Note, no need to recommend isdataat, I know there is data within 1024
> > > > > bytes past the previous content match.
> > > > >
> > > > > Does NOT work:
> > > > >    uricontent:"/login.php"; fast_pattern; content:"|0d 0a|Cookie\: ";
> > > > > nocase; http_header; content:"ieatbugs="; within:1024;
> > > > >
> > > > > Does work:
> > > > >    uricontent:"/login.php"; fast_pattern; content:"|0d 0a|Cookie\: ";
> > > > > nocase; content:"ieatbugs="; within:1024;
> > > > >
> > > > > Comments/insight appreciated.
> > > > >
> > > > > -evilghost
> > > > >
> > > > > ------------------------------------------------------------------------------
> > > > > Download Intel&#174; Parallel Studio Eval
> > > > > Try the new software tools for yourself. Speed compiling, find bugs
> > > > > proactively, and fine-tune applications for parallel performance.
> > > > > See why Intel Parallel Studio got high marks during beta.
> > > > > http://p.sf.net/sfu/intel-sw-dev
> > > > > _______________________________________________
> > > > > Snort-sigs mailing list
> > > > > Snort-sigs () lists sourceforge net
> > > > > https://lists.sourceforge.net/lists/listinfo/snort-sigs
> > > ------------------------------------------------------------------------------
> > > Download Intel&#174; Parallel Studio Eval
> > > Try the new software tools for yourself. Speed compiling, find bugs
> > > proactively, and fine-tune applications for parallel performance.
> > > See why Intel Parallel Studio got high marks during beta.
> > > http://p.sf.net/sfu/intel-sw-dev
> > > _______________________________________________
> > > Snort-sigs mailing list
> > > Snort-sigs () lists sourceforge net
> > > https://lists.sourceforge.net/lists/listinfo/snort-sigs

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Snort-sigs mailing list
Snort-sigs () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-sigs