Re: [Emerging-Sigs] Touched by a proxy: thoughts on urilen?

[L0rd Ch0de1m0rt <l0rdch0de1m0rt () gmail com>]

Hello. I have reproduced this issue on Snort 2.9.4.  Having a full URI
(e.g. http://www.bigbadmalwarecandc.com) after the HTTP request verb
somehow messes up the processing of the http_uri buffer (perhaps more than
this I do not know) in undesirable ways.  Specifically, I have unearthed
issues where relative content matches in the http_uri buffer does not work
as expected and can lead to Snort IDS bypass in proxied environments.  As
many people use Snot to inspect proxied environment traffic, my management
and I consider this to be fairly serious although I haven't calculated a
CVSS number yet.

I suspect that the HTTP Pre Processor has a number of "issues" in detecting
traffic in proxied environments (see previous note about urilen) and
perhaps such environments were not very rigorously tested with Snort but
that is just speculation and I in no way mean to besmirch the snort
developers.

These issues have been described in detail in this thread, along with
applicable rules and even a pcap so that whosoever wishes to re-produce may
do so.

I am copying snort-devel for good measure even though they should already
know about this.

Thanks again for your help Joel.  I will await confirmation of issues and
scheduled patch/fix dates if applicable.

Cheers,

-Lord C.

On Fri, Feb 22, 2013 at 4:07 PM, L0rd Ch0de1m0rt
<l0rdch0de1m0rt () gmail com>wrote:

> Hello.  Thanks Joel.  I stopped posting to the snort lists a long time ago
> when I realized I was being moderated. But I do appreciate what you do and
> the snort-dev folks do.  Next pint I have is for you guys if I don't get
> moderated (cut off), haha.
>
> If this is a Snort IDS bypass bug, can I get a CVE award?
>
> /me is stopping drunk dialing the list now.   sorry all for the noise but
> I do think this issue is serious business.
>
> Cheers,
>
> -Lord C.
>
>
> On Fri, Feb 22, 2013 at 3:42 PM, Joel Esler <jesler () sourcefire com> wrote:
>
> > Snort-dev *is* involved ;)
> >
> > As for your posts, I don't see any posts from you in the queue, so I
> > don't know what you mean your posts are restricted.
> >
> > I'll get this over to devel.
> >
> >
> > J
> >
> >
> > On Feb 22, 2013, at 4:35 PM, L0rd Ch0de1m0rt <l0rdch0de1m0rt () gmail com>
> > wrote:
> >
> > Hello.  See attached pcap my mate just forwarded me.  This is what was
> > used for testing.  Maybe you can re-produce.
> >
> > Looks like the 'http://&apos; in the URI due to proxy usage is troubling up
> > the http pre processor very badly.  Maybe we should get snort-dev involved?
> > I would but my posts are restricted.
> >
> > Next round on me, chaps!
> >
> > Cheers,
> >
> > -Lord C.
> >
> > On Fri, Feb 22, 2013 at 1:38 PM, L0rd Ch0de1m0rt <
> > l0rdch0de1m0rt () gmail com> wrote:
> >
> > > Hello.  Thanks for the feedback all.  Let me try to be clearer since the
> > > issue being talk about now is a little different from what was first
> > > brought up.
> > >
> > > Here is the traffic (http proxy GET request), with normal headers,
> > > trailing CRLFCRLF, etc. (not shown):
> > >
> > > GET http://begopektdssite12.info/icon.php HTTP/1.1
> > >
> > > Rule that alerts:
> > >
> > > alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET
> > > CURRENT_EVENTS BegOpEK - TDS - icon.php"; flow:established,to_server;
> > > content:"/"; http_uri; content:"icon.php"; distance:0; http_uri;
> > > classtype:trojan-activity; sid:2015789xxx; rev:1;)
> > >
> > > Rule that doesn't alert (but should in my HO):
> > >
> > > alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET
> > > CURRENT_EVENTS BegOpEK - TDS - icon.php"; flow:established,to_server;
> > > content:"/"; http_uri; content:"icon.php"; distance:0; within:8;http_uri; classtype:trojan-activity; 
> > > sid:2015789xxx; rev:1;)
> > >
> > > Cheers,
> > >
> > > -Lord C.
> > >
> > >
> > >
> > > On Fri, Feb 22, 2013 at 12:56 PM, Joel Esler <jesler () sourcefire com>wrote:
> > >
> > > > Sure,  because you are moving between buffers.  That's not what I
> > > > thought was being said.
> > > >
> > > >
> > > > --
> > > > *Joel Esler*
> > > > Sent from my iPhone 
> > > >
> > > > On Feb 22, 2013, at 1:55 PM, William Metcalf <william.metcalf () gmail com>
> > > > wrote:
> > > >
> > > > Not with within:8; it won't work.
> > > >
> > > > Regards,
> > > >
> > > > Will
> > > >
> > > > On Feb 22, 2013, at 12:42 PM, Joel Esler <jesler () sourcefire com> wrote:
> > > >
> > > > Without the modifier the rule will check all buffers. So it'll still
> > > > work, just more fp prone.
> > > >
> > > > --
> > > > *Joel Esler*
> > > > Sent from my iPhone 
> > > >
> > > > On Feb 22, 2013, at 1:14 PM, L0rd Ch0de1m0rt <l0rdch0de1m0rt () gmail com>
> > > > wrote:
> > > >
> > > > Hello. Oh thanks for pointing that out Will (very important!).  In
> > > > fact, I did have 'http_uri' on the second content match in my testing, I
> > > > just did not grab it when I did the copy & paste.  It should read:
> > > >
> > > > content:"/"; http_uri; content:"icon.php"; distance:0; within:8;
> > > > http_uri;
> > > >
> > > > If I do 'distance:0; within:100;' it alerts so I think the Snort is not
> > > > being thorough or complete with the match and getting hung up on the '/' in
> > > > the 'http://&apos; part of the URI buffer and not continuing and checking
> > > > other '/' with regard to the relative content matches that depend on it.
> > > >
> > > > I hope I am wrong about this because this could be serious for relative
> > > > matching in URI buffers.
> > > >
> > > > Cheers,
> > > >
> > > > -Lord C.
> > > >
> > > > On Fri, Feb 22, 2013 at 11:50 AM, Will Metcalf <
> > > > wmetcalf () emergingthreatspro com> wrote:
> > > >
> > > > > you have to add http_uri; to the second content match otherwise this
> > > > > construct fails silently on snort.
> > > > >
> > > > >  Regards,
> > > > >
> > > > > Will
> > > > >
> > > > >
> > > > > On Fri, Feb 22, 2013 at 11:39 AM, L0rd Ch0de1m0rt <
> > > > > l0rdch0de1m0rt () gmail com> wrote:
> > > > >
> > > > > > Hello.  Thank you Harry.  This is good to know that I am not the only
> > > > > > one with consternation.  To deal with this, I thought I might remove
> > > > > > 'urilen' and try this to work-around it (still keeping the example of
> > > > > > sid:2015789).  This example is trivial (it could be just a single content
> > > > > > match) but the behaviour could have serious issue for other rules that need
> > > > > > to use distance and within on the URI:
> > > > > >
> > > > > > content:"/"; http_uri; content:"icon.php"; distance:0; within:8;
> > > > > >
> > > > > > For some reason, this does not alert with the request is:
> > > > > >
> > > > > > GET http://begopektdssite12.info/icon.php HTTP/1.1
> > > > > >
> > > > > > Is the http pre processor getting confused?  Is the content match for
> > > > > > "/" being "one-and-done non-greedy" as my friend put it? If I remove the
> > > > > > 'within' it alerts me and the http pre processor is being used.
> > > > > >
> > > > > > Can anyone else re-produce?
> > > > > >
> > > > > > Now I am more seriously consternated!
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > -Lord C.
> > > > > >
> > > > > >
> > > > > > On Fri, Feb 22, 2013 at 9:08 AM, harry.tuttle <harry.tuttle () zoho com>wrote:
> > > > > >
> > > > > > > You are absolutely correct. Urilen and a ^ at the beginning of uri
> > > > > > > pcre are major stumbling blocks to proxies.
> > > > > > >
> > > > > > > Having said that, some malware of course won't even try to use a
> > > > > > > proxy, so it is difficult to know on which rules it is actually a problem.
> > > > > > > When I am sandboxing and developing rules, I always run my sample both with
> > > > > > > and without a proxy set to know for sure.
> > > > > > >
> > > > > > > I added the "proxy vs direct" and related "HTTP Library Variation"
> > > > > > > sections to the wiki a while back, but I'm sure those could use some
> > > > > > > elaboration.
> > > > > > > http://doc.emergingthreats.net/bin/view/Main/SnortSigs101
> > > > > > >
> > > > > > > Regards,
> > > > > > > Harry
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > ---- On Fri, 22 Feb 2013 06:50:07 -0800 L0rd Ch0de1m0rt  wrote ----
> > > > > > >
> > > > > > > > Hello.  I am posting here instead of snort-sigs because that list
> > > > > > > is moderated and not all submissions get sent out.  Also, I consider
> > > > > > > Emerging Threats a good (better?) community and value their insights.
> > > > > > > > A friend and I recently had a discussions about the 'urilen' option
> > > > > > > in Snort.  I did some testing on a Snort 2.9 install and it seems that that
> > > > > > > the engine may not take into account proxy using.  I am guess that this is
> > > > > > > as designed but it could have non trivial implication for the many Snort
> > > > > > > users that have a proxy that is in use on their network and have the Snort
> > > > > > > installed to look at the traffic.  Take this Emerging Threats rule for a
> > > > > > > example:
> > > > > > > > alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET
> > > > > > > CURRENT_EVENTS BegOpEK - TDS - icon.php"; flow:established,to_server;
> > > > > > > content:"/icon.php"; urilen:9; classtype:trojan-activity; sid:2015789;
> > > > > > > rev:1;)
> > > > > > > > Instead of a request line looking like this:
> > > > > > > >
> > > > > > > > GET /icon.php HTTP/1.1
> > > > > > > >
> > > > > > > > what if the client was configured to use a proxy so the request
> > > > > > > looked like this:
> > > > > > > > GET http://begopektdssite12.info/icon.php HTTP/1.1
> > > > > > > >
> > > > > > > > I do not think the rule would fire.  I assumed that the http pre
> > > > > > > processor would account for proxy usage and compensate but it does not in
> > > > > > > snort with my testing.  I tried "urilen:9,norm" and "urilen:9,raw" but it
> > > > > > > still did not alert me.
> > > > > > > > If my tests are true, this means that alot of the Emerging Threats
> > > > > > > sigs that use urilen may not work as desired in proxy environments.
> > > > > > > > Am I wrong on this?  Is there a proxy configurations in Snort that
> > > > > > > I need to use?  Please tell me I am wrong.
> > > > > > > > Does Suricata do a better job or give the option to "normalize" the
> > > > > > > proxy request if you just want to match on URI after the domain part?
> > > > > > > > Cheers,
> > > > > > > >
> > > > > > > > Lord C.
> > > > > > > > _______________________________________________
> > > > > > > > Emerging-sigs mailing list
> > > > > > > > Emerging-sigs () lists emergingthreats net
> > > > > > > > http://lists.emergingthreats.net/mailman/listinfo/emerging-sigs
> > > > > > > >
> > > > > > > > Support Emerging Threats! Subscribe to Emerging Threats Pro
> > > > > > > http://www.emergingthreatspro.com
> > > > > > > > The ONLY place to get complete premium rulesets for all versions of
> > > > > > > Suricata and Snort 2.4.0 through Current!
> > > > > >
> > > > > > _______________________________________________
> > > > > > Emerging-sigs mailing list
> > > > > > Emerging-sigs () lists emergingthreats net
> > > > > > http://lists.emergingthreats.net/mailman/listinfo/emerging-sigs
> > > > > >
> > > > > > Support Emerging Threats! Subscribe to Emerging Threats Pro
> > > > > > http://www.emergingthreatspro.com
> > > > > > The ONLY place to get complete premium rulesets for all versions of
> > > > > > Suricata and Snort 2.4.0 through Current!
> > > > _______________________________________________
> > > > Emerging-sigs mailing list
> > > > Emerging-sigs () lists emergingthreats net
> > > > http://lists.emergingthreats.net/mailman/listinfo/emerging-sigs
> > > >
> > > > Support Emerging Threats! Subscribe to Emerging Threats Pro
> > > > http://www.emergingthreatspro.com
> > > > The ONLY place to get complete premium rulesets for all versions of
> > > > Suricata and Snort 2.4.0 through Current!
> > <icon.php.test.pcap>
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb

_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about Snort!