Re: Question about Content-Disposition, Content-Type, etc. and http_header buffer

[Joel Esler <jesler () sourcefire com>]

Mike,

I understand what you mean, however the http inspect preprocessor should handle multipart headers.  There's a format 
that they must adhere to in the RFC as well for that, and I know we handle that as I had to do some testing on that 
awhile back to ensure that we not only detect but handle that.    So I'd ask you to verify that this is, indeed the 
case.  (You say "I believe", so I'm just being sure)

That being said, yes, we are rewriting http_inspect right now and multipart headers are one of the feature 
requirements. (as the present http_inspect handles it).  I'll ask devel to take a look at this, but my testing and 
understanding is that we handle it.  

I know you said you can't get it, but that's why we always ask for a pcap.  We can't recreate every network in the 
world, and not all devices adhere to RFCs correctly.  Heck, some vendors don't adhere to their own RFC's.  Development 
will ask me for a pcap from you, as our pcaps that are testing with show we handle this multipart (if that's what it 
really is) scenario correctly.

I'm not trying to be difficult, and I'm not arrogant enough to think that there's some scenario out that there that we 
might not handle.  We're all humans, we know it probably exists, however, given the scenario that you describe, and 
from my experience, that shouldn't be a problem.

--
Joel Esler
Senior Research Engineer, VRT
OpenSource Community Manager
Sourcefire


On Oct 17, 2012, at 1:04 PM, Mike Cox <mike.cox52 () gmail com> wrote:

> Hi Joel,
>
> I appreciate your timely response and feedback.
>
> I believe the issue here is that HTTP headers normally seen in the initial HTTP data are being sent in subsequent 
> packets (as a result of multipart/form-data POSTS) and formatted differently from the standard HTTP header.  I hope I 
> made this clear in previous emails but some simple Google searches would also help illuminate what I'm talking about. 
>  I'd rather not get in to a RFC argument with you (if you really want to we can but, apparently this is a standard 
> format and accepted by both modern browsers and servers).
>
> My concern here is Snort rules that are looking for data normally in the HTTP headers (using the http_header buffer) 
> do not work in the case of some multipart/form data POSTS.  This means that snort rules are not alerting as intended 
> (and introduces an IDS bypass avenue, albeit somewhat paltry in terms of eloquence but nonetheless functional).
>
> Does Sourcefire/VRT,etc. intend to account for this in future http-inspect capabilities or should we modify our 
> current rules to do inefficient, global content matches?  Or is there another solution that I am missing here?  What 
> can we do *now*?
>
> Thanks.
>
> -Mike Cox
>
>
> On Wed, Oct 17, 2012 at 10:51 AM, Joel Esler <jesler () sourcefire com> wrote:
> Well, from the looks of the "dump" (or whatever you sent below) it looks like there are multiple carriage returns in 
> there.
>
> If the "header" of an http session has |0d 0a 0d 0a|, according to the RFC which states that character sequence 
> indicates the end of the header, and the start of the packet data.  In which case the http_inspect preprocessor will 
> move onto the http_client_data buffer.
>
> Hence why I asked for the pcap, to see if this is true.  We can create a pcap that looks like yours below, but I know 
> the exact behavior of the preprocessor in this case.
>
> If you wanted to test it, you could do a 
> content:"Content-Type"; http_header; content:"Content-Disposition"; http_header; distance:0;
>
> which you indicated doesn't work.
>
> then test by doing:
>
> content:"Content-Type"; http_header; content:"Content-Disposition"; http_client_body;
>
> and if that fires, then there's your problem.  Broken RFC adherence by whatever program is generating that traffic.
>
> --
> Joel Esler
> Senior Research Engineer, VRT
> OpenSource Community Manager
> Sourcefire
>
>
> On Oct 17, 2012, at 11:45 AM, Mike Cox <mike.cox52 () gmail com> wrote:
>
> > Hey Joel, thanks for the quick response.  Right now I don't have a pcap ... the info I got was reproduced from what 
> > I saw on a screen while helping our guys/gals track down an issue/RCA in the "Temptest" (SOC) room and I can't get 
> > any data out of there and if I could, it would be breaking many laws (well, internal "laws" but there could be 
> > international intel impact and NDA lawsuits if you know what I mean :)
> >
> > However, I think that the information provided is sufficient to illustrate this issue and it should be trivial for 
> > VRT to create necessary pcaps for their testing, if that is what you want.
> >
> > I can cook up a pcap but as earlier implied, I don't think it is is necessary for me to provide this at this point 
> > and I certainly do not want to insult VRT by spoon-feeding them data that they can easily and more completely create 
> > on their own with minimal effort.
> >
> > I believe that I have outlined the issue and provided the necessary data to accurately describe the problem.  Even 
> > without a pcap, I think the info provided is sufficient for a good discussion.  Has anyone else experienced this 
> > challenge with their snort installs?  If so, how do you deal with this nuance of HTTP and the current state of 
> > http-inspect?
> >
> > Please let me know if you are unclear on anything or need more information regarding this issue.
> >
> > Thanks all.
> >
> > -Mike Cox
> >
> >
> >
> > On Tue, Oct 16, 2012 at 4:51 PM, Joel Esler <jesler () sourcefire com> wrote:
> > Can you send me a pcap?
> >
> > --
> > Joel Esler
> > Senior Research Engineer, VRT
> > OpenSource Community Manager
> > Sourcefire
> >
> > On Oct 16, 2012, at 5:18 PM, Mike Cox <mike.cox52 () gmail com> wrote:
> >
> > > I've noticed that in some multipart/form-data POSTs, data that is normally in the HTTP header gets sent in the body 
> > > of the message and not parsed by http-inspect as part of the http_header buffer.  Specifically, the headers 
> > > "Content-Type", "Content-Disposition", and "Content-Transfer-Encoding", although there could be others.  For 
> > > example:
> > >
> > > POST /blackhole/safe.php HTTP/1.1
> > > Host: snort.org
> > > Content-Type: multipart/form-data, boundary=---dG91Y2hteXNub3J0
> > > Content-Length: 8675309
> > >
> > > ---dG91Y2hteXNub3J0
> > > Content-Disposition: form-data; name="name"
> > >
> > > Joshua
> > > ---dG91Y2hteXNub3J0
> > > Content-Disposition: form-data; name="play_a_game"
> > >
> > > True
> > > ---dG91Y2hteXNub3J0
> > > Content-Disposition: form-data; name="file"; filename="GLOBAL_THERMONUCLEAR_WAR.pdf"
> > > Content-Type: application/pdf
> > > Content-Transfer-Encoding: binary
> > > ...
> > >
> > > So a snort rule looking for a specific filename in a Content-Disposition header wouldn't match if it were written 
> > > as you would expect it to be written.  For example, this wouldn't match the above:
> > >
> > > alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Bad PDF File Upload"; flow:established,to_server; 
> > > content:"Content-Disposition"; http_header; content:"filename="; distance:0; http_header; content:".pdf"; 
> > > distance:0; within:100; http_header; sid:1234567;)
> > >
> > > What is the best way to match this and not incur the overhead of using global content matches?  Is there a plan for 
> > > the http-inspect pre-processor to account for this?
> > >
> > > Thanks.
> > >
> > > -Mike Cox
> > > ------------------------------------------------------------------------------
> > > Everyone hates slow websites. So do we.
> > > Make your web apps faster with AppDynamics
> > > Download AppDynamics Lite for free today:
> > > http://p.sf.net/sfu/appdyn_sfd2d_oct_______________________________________________
> > > Snort-sigs mailing list
> > > Snort-sigs () lists sourceforge net
> > > https://lists.sourceforge.net/lists/listinfo/snort-sigs
> > > http://www.snort.org
> > >
> > >
> > > Please visit http://blog.snort.org for the latest news about Snort!

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct

_______________________________________________
Snort-sigs mailing list
Snort-sigs () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-sigs
http://www.snort.org


Please visit http://blog.snort.org for the latest news about Snort!