Re: BUG: corner case involving http_cookie

[Will Metcalf <william.metcalf () gmail com>]

How about this one, using the same pcap. I have both client and server
flow_depth set to 0.

#test 72 http_cookie + byte_test.
#hmm interesting using http_cookie in combination with byte_test seems
to fail always. Removing either the byte_test check or the http_cookie
modifier will cause this sig to fire.  Notice that the byte_test
#check isn't even relative to the content match.
#
#file oisfsearchnums.pcap
alert tcp any any -> any any (msg:"e6504ae48c99f09df7f58996aacbb6b0 +
http_cookie + byte_test"; content:"e6504ae48c99f09df7f58996aacbb6b0";
http_cookie; byte_test:2,=,12,596,string,dec; classtype:bad-unknown;
sid:72; rev:1;)


Regards,

Will

On Wed, Mar 10, 2010 at 1:54 PM, Will Metcalf <william.metcalf () gmail com> wrote:
> I can see the need for the cut-off.  You have to make compromises
> between accuracy and performance this tends to be the nature of the
> beast with IDS's,  and sure there will always be evasions abut the
> dynamic flow_depth thing still isn't making much sense to me and/or at
> least this behavior should be documented somewhere.  Did I just miss
> something in the readme?  Btw there is a typo both in the manual and
> the README regarding client_flow_depth.  "It primarily eliminates
> Snort fro inspecting larger HTTP Cookies that appear at the end of
> many client request Headers."
>
>
> Regards,
>
> Will
>
> On Wed, Mar 10, 2010 at 1:18 PM, Steven Sturges
> <steve.sturges () sourcefire com> wrote:
> > There are different settings for the requests & responses.
> > See client_flow_depth and server_flow_depth in the readme/manual.
> >
> > Setting either of the flow depth values to 0 would inspect
> > all of the respective request or response, but incurs a
> > pretty big performance hit.
> >
> > And yes, the default (in code) is 300 for both.
> >
> > Will Metcalf wrote:
> > > Ahhh ok so I want to make sure I understand correctly in 2.8.5 the
> > > default 300 byte cutoff is not only applied to http_responses but also
> > > http_requests (README.http_inspect) if there are http normalizations.
> > > Is there not still a potential evasion here based on the tests below
> > > as packet being normalized or potentially matching on one or more
> > > rules can be fairly unpredictable?  Would your recommended
> > > configuration then be to set client_flow_depth to 0 in the
> > > http_inspect preproc for 2.8.5 users if they wanted to avoid the
> > > potential evasion?
> > >
> > > Regards,
> > >
> > > Will
> > >
> > > On Wed, Mar 10, 2010 at 12:03 PM, Steven Sturges
> > > <steve.sturges () sourcefire com> wrote:
> > > > Seems that what is really coming into play is the flow depth.
> > > > That is going to limit how much of the raw data is searched
> > > > with the pattern matcher.
> > > >
> > > > To summarize how it works with 2.8.6:
> > > >
> > > > 1) If HTTP normalizes headers and cookies, the amount of raw data
> > > > searched by fast pattern matcher is determined by client (or server)
> > > > flow depth settings.
> > > >
> > > > 2) If there are no HTTP normalizations, flow depth is not applied, so
> > > > all of the raw data is searched.
> > > >
> > > > With 2.8.5, in 1) above, the flow depth was applied only if there were
> > > > rules searched for one of those HTTP normalized buffers.
> > > >
> > > > By default, client flow depth is 0 -- which is what Will is probably
> > > > hitting in his tests.
> > > >
> > > > Alerts w/ combinations of rules & different flow depths in 2.8.6:
> > > >
> > > > Default flow depth (0)
> > > > SID 59, SID 68 enabled
> > > > 1 alert (sid 59)
> > > >
> > > > Default flow depth (0)
> > > > SID 68 enabled
> > > > 0 alerts
> > > >
> > > > [Same as above for client_flow_depth 300]
> > > >
> > > > Client flow depth (1460)
> > > > SID 59, SID 68 enabled
> > > > 3 alerts (sid 59, sid 68x2)
> > > >
> > > > Client flow depth (1460)
> > > > SID 68 enabled
> > > > 2 alerts (sid 68x2)
> > > >
> > > > -steve
> > > >
> > > > Matt Jonkman wrote:
> > > > > Appreciate the clarification Steve. But I'm concerned, this will make
> > > > > hundreds if not a few thousand rules not work correctly in our set and
> > > > > in vrt/snort gpl. If someone has http_inspect on in a recent snort, but
> > > > > does NOT have ALL of their http related rules converted to the new form
> > > > > using the http_* modifiers (which we have none converted) then they're
> > > > > going to have massive problems, no? I think a lot of people are missing
> > > > > a lot of things right now.
> > > > >
> > > > > Is there a way to make http_inspect not do this? We have a lot of sigs
> > > > > that won't work on normalized data, for instance the recent trojan sigs
> > > > > that look at the order of the parameters in an http post for uniqieness.
> > > > > With the normalized only we can't see that can we?
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Matt
> > > > >
> > > > > On 3/10/10 10:26 AM, Will Metcalf wrote:
> > > > > > > 1) Use http_cookie in the rule as you note in the case that works.
> > > > > > Right so the reason that I cc'd the emerging list is that they are not
> > > > > > using the http_* modifiers to maintain compatibility with older
> > > > > > versions of snort.  Shouldn't this buffer at least be available to
> > > > > > match on via rawbytes  which is what would be consistent with what you
> > > > > > have done with telnet and dcerpc.  Does this also not add a potential
> > > > > > evasion method if this is the intended behavior, one that perhaps
> > > > > > VRT/ET should be made aware of.  I haven't gone through very many of
> > > > > > the sigs but since the normalized buffer begins with the "Cookie:"
> > > > > > instead of the value isn't there a now a potential evasion if I can
> > > > > > get another sig to trip on the same packet as say sid 2136.
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Will
> > > > > >
> > > > > > On Wed, Mar 10, 2010 at 8:15 AM, Steven Sturges
> > > > > > <steve.sturges () sourcefire com> wrote:
> > > > > > > Will--
> > > > > > >
> > > > > > > In the 2nd rule the only content is an HTTP cookie.  Without using
> > > > > > > http_cookie, it would try to match the raw data.
> > > > > > >
> > > > > > > With HTTP Inspect enabled, it is separating the headers, cookie,
> > > > > > > method, etc from the raw data, hence all of the modifier keywords
> > > > > > > that you can use with content -- can use more than one together.
> > > > > > >
> > > > > > > Without specifying http_cookie in the rule and when HTTP Inspect
> > > > > > > enabled (and cookie inspection enabled in 2.8.6), if the pattern
> > > > > > > matcher searches any of the HTTP buffers, it doesn't search the raw
> > > > > > > data.  If no rules use the specific HTTP buffers or there are no HTTP
> > > > > > > buffers, the pattern matcher will search the raw data, which is why
> > > > > > > the rule byte itself (without http_cookie) works.
> > > > > > >
> > > > > > > This was done to avoid going over the same data twice in the pattern
> > > > > > > matcher, and it is working as it is designed.
> > > > > > >
> > > > > > > Two options:
> > > > > > >
> > > > > > > 1) Use http_cookie in the rule as you note in the case that works.
> > > > > > >
> > > > > > > 2) Turn off HTTP Inspect (not realistic)
> > > > > > >
> > > > > > > Cheers
> > > > > > > -steve
> > > > > > >
> > > > > > > Will Metcalf wrote:
> > > > > > > > hmmm I don't think so.  Look at first test.  both rules fire.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > >
> > > > > > > > Will
> > > > > > > >
> > > > > > > > On Tue, Mar 9, 2010 at 10:31 PM, beenph <beenph () gmail com> wrote:
> > > > > > > > > I will try a wild guess, what is your event_queue size like?
> > > > > > > > >
> > > > > > > > > Its probably a bug or something  that need clarification regarding
> > > > > > > > > http_cookie and http_inspect, but mabey http_cookie enable a modifier
> > > > > > > > > in http_inspect that alter alerting behavior when event_queue is at 1
> > > > > > > > > (since i guess both "alerts" are part of the same normalized http
> > > > > > > > > stream)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > -elz
> > > > > > > > > ps: didin't run the pcap and rules test.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, Mar 9, 2010 at 11:15 PM, Will Metcalf <william.metcalf () gmail com> wrote:
> > > > > > > > > > failing to use the http_cookie modifier on a rule where there is
> > > > > > > > > > another rule that matches the same packet makes a rule that should
> > > > > > > > > > fire fail.
> > > > > > > > > >
> > > > > > > > > > src/snort -V
> > > > > > > > > >
> > > > > > > > > >   ,,_     -*> Snort! <*-
> > > > > > > > > >  o"  )~   Version 2.8.5.3 (Build 124)
> > > > > > > > > >   ''''    By Martin Roesch & The Snort Team:
> > > > > > > > > > http://www.snort.org/snort/snort-team
> > > > > > > > > >           Copyright (C) 1998-2009 Sourcefire, Inc., et al.
> > > > > > > > > >           Using PCRE version: 7.8 2008-09-05
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > src/snort -k none -q -A console -c etc/snort.conf -l ./ -r oisfsearchnums.pcap
> > > > > > > > > >
> > > > > > > > > > #this combo works
> > > > > > > > > > #alert tcp any any -> any any (msg:"http_client_body";
> > > > > > > > > > content:"searchword="; uricontent:"/index.php"; nocase;
> > > > > > > > > > classtype:bad-unknown; sid:59; rev:1;)
> > > > > > > > > > #alert tcp any any -> any any (msg:"http_cookie match ";
> > > > > > > > > > content:"e6504ae48c99f09df7f58996aacbb6b0=120e494ce857d6ceeef89f9678d4d703";
> > > > > > > > > > http_cookie; classtype:bad-unknown; sid:68; rev:1;)
> > > > > > > > > > #
> > > > > > > > > > #03/07-21:19:54.242506  [**] [1:59:1] http_client_body [**]
> > > > > > > > > > [Classification: Potentially Bad Traffic] [Priority: 2] {TCP}
> > > > > > > > > > 192.168.100.17:38111 -> 96.43.130.5:80
> > > > > > > > > > #03/07-21:19:54.242506  [**] [1:68:1] http_cookie match  [**]
> > > > > > > > > > [Classification: Potentially Bad Traffic] [Priority: 2] {TCP}
> > > > > > > > > > 192.168.100.17:38111 -> 96.43.130.5:80
> > > > > > > > > > #03/07-21:19:54.364173  [**] [1:68:1] http_cookie match  [**]
> > > > > > > > > > [Classification: Potentially Bad Traffic] [Priority: 2] {TCP}
> > > > > > > > > > 192.168.100.17:38111 -> 96.43.130.5:80
> > > > > > > > > >
> > > > > > > > > > #the second rule does not fire
> > > > > > > > > > #alert tcp any any -> any any (msg:"http_client_body + depth";
> > > > > > > > > > content:"searchword="; uricontent:"/index.php"; nocase;
> > > > > > > > > > classtype:bad-unknown; sid:59; rev:1;)
> > > > > > > > > > #alert tcp any any -> any any (msg:"http_cookie match";
> > > > > > > > > > content:"e6504ae48c99f09df7f58996aacbb6b0=120e494ce857d6ceeef89f9678d4d703";
> > > > > > > > > > classtype:bad-unknown; sid:68; rev:1;)
> > > > > > > > > > #
> > > > > > > > > > #03/07-21:19:54.242506  [**] [1:59:1] http_client_body + depth [**]
> > > > > > > > > > [Classification: Potentially Bad Traffic] [Priority: 2] {TCP}
> > > > > > > > > > 192.168.100.17:38111 -> 96.43.130.5:80
> > > > > > > > > >
> > > > > > > > > > #this rule fires when used on it's own.
> > > > > > > > > > #alert tcp any any -> any any (msg:"http_cookie match";
> > > > > > > > > > content:"e6504ae48c99f09df7f58996aacbb6b0=120e494ce857d6ceeef89f9678d4d703";
> > > > > > > > > > classtype:bad-unknown; sid:68; rev:1;)
> > > > > > > > > > #
> > > > > > > > > > #03/07-21:19:54.242506  [**] [1:68:1] http_cookie match [**]
> > > > > > > > > > [Classification: Potentially Bad Traffic] [Priority: 2] {TCP}
> > > > > > > > > > 192.168.100.17:38111 -> 96.43.130.5:80
> > > > > > > > > > #03/07-21:19:54.364173  [**] [1:68:1] http_cookie match [**]
> > > > > > > > > > [Classification: Potentially Bad Traffic] [Priority: 2] {TCP}
> > > > > > > > > > 192.168.100.17:38111 -> 96.43.130.5:80
> > > > > > > > > >
> > > > > > > > > > ------------------------------------------------------------------------------
> > > > > > > > > > Download Intel&#174; Parallel Studio Eval
> > > > > > > > > > Try the new software tools for yourself. Speed compiling, find bugs
> > > > > > > > > > proactively, and fine-tune applications for parallel performance.
> > > > > > > > > > See why Intel Parallel Studio got high marks during beta.
> > > > > > > > > > http://p.sf.net/sfu/intel-sw-dev
> > > > > > > > > > _______________________________________________
> > > > > > > > > > Snort-devel mailing list
> > > > > > > > > > Snort-devel () lists sourceforge net
> > > > > > > > > > https://lists.sourceforge.net/lists/listinfo/snort-devel
> > > > > > > > ------------------------------------------------------------------------------
> > > > > > > > Download Intel&#174; Parallel Studio Eval
> > > > > > > > Try the new software tools for yourself. Speed compiling, find bugs
> > > > > > > > proactively, and fine-tune applications for parallel performance.
> > > > > > > > See why Intel Parallel Studio got high marks during beta.
> > > > > > > > http://p.sf.net/sfu/intel-sw-dev
> > > > > > > > _______________________________________________
> > > > > > > > Snort-devel mailing list
> > > > > > > > Snort-devel () lists sourceforge net
> > > > > > > > https://lists.sourceforge.net/lists/listinfo/snort-devel
> > > > > > ------------------------------------------------------------------------------
> > > > > > Download Intel&#174; Parallel Studio Eval
> > > > > > Try the new software tools for yourself. Speed compiling, find bugs
> > > > > > proactively, and fine-tune applications for parallel performance.
> > > > > > See why Intel Parallel Studio got high marks during beta.
> > > > > > http://p.sf.net/sfu/intel-sw-dev
> > > > > > _______________________________________________
> > > > > > Snort-devel mailing list
> > > > > > Snort-devel () lists sourceforge net
> > > > > > https://lists.sourceforge.net/lists/listinfo/snort-devel

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel