Re: IDS detection approaches

["Sec urity" <security () brvenik com>]

On 10/9/07, Nelson Brito <nbrito () sekure org> wrote:
> > There are few commercial IPS products that stand up to informed
> > scrutiny as well as Snort. Too bad many commercial black box vendors
> > are allowed to claim something other than reality. What leads you to
> > believe that other vendors are not simple signature engines or are
> > some how superior?
>
> I didn't say that any other IDS/IPS do a superior or a super detection, I
> just said that PATTERN MATCHING is so "cat X | grep X", even if you are
> naming it as regex, just to give a new name. BTW, regex do a better PATTERN
> MATCHING, anyway...
>
> I didn't claim the death of pattern, I'm just saying that there are better
> ways to understand and detect an attack to vulnerability, using mixed
> approaches and new technologies, and I do prefer to detect an attack to
> vulnerabilities by using vulnerabilities signature algorithms instead of
> PATTERN MATCHING.

My apologies, the use of SNORT in the context it was led me to a
different conclusion. More below and that will be all for me, regards.

> As far as I know, there are too many ways to evade any IDS/IPS, but how many
> ways work in just one IDS/IPS defines which one is the better approach, and
> I can tell you that any IDS/IPS that has all their computational resources
> to do PATTERN MATCHING is more susceptible to well known and unknown attacks
> addressing PATTERN MATCHING weakness.
>
> Lets check the the following signature:
> alert udp $EXTERNAL_NET any -> $HOME_NET 1434 (\
>         msg:"MS-SQL version overflow attempt";\
>         flowbits:isnotset,ms_sql_seen_dns; dsize:>100; content:"|04|";
> depth:1;\
>         reference:bugtraq,5310;\
>         reference:cve,2002-0649;\
>         reference:nessus,10674;\
>         classtype:misc-activity; sid:2050; rev:8;\
> )
>
> Does it protect you agains SLAMMER? Yes, it does... But it opens a door to
> do a False Positive attack.
>
> There are four main variables for this vulnerability work:
> - Protocol UDP
> - Port 1434
> - SQL Request 0x04 (first byte in the payload)
> - Buffer size > 96, with no NULL byte and being the 96th byte the return
> address to be overwrite.
>
> If you miss any of them you are not able to exploit the SQL Server.
>
> FACT!!!
>
> We can see that the previous signature (PATTERN MATCHING) looks for:
> protocol UDP, port 1434, first byte in payload equals to 0x04 and the size
> of 100 (buffer of 96 bytes + and the new return addres of 4 bytes)... But
> there is no check to ignore payload with 96 NULL bytes.
>
> FACT!!!
>
> What if you send 1,000,000 packets with the following payload to port
> 1434/UDP?
>
>         "\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
>         "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
>         "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
>         "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
>         "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
>         "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
>         "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
>
> I can tell you: 1,000,000 events... Sorry, false events, as known as false
> positives. How bad could it be to your PATTERN MATCHING? I can tell you:
> - HD exhaust
> - 100% of CPU
> - Memory exhaust
> - etc etc etc
>
> FACT!!!
>
> Some of "smart" IDS/IPS will shutdown the signature to prevent itself of
> rebooting of any other damage, just like that.

Well...

1) If it is a real concern you should have configured a default global
threshold or the latency thresholds if inline. This default global
would have kicked in and prevent this kind of resource exhaustion.

2) Your attempt at resource exhaustion is still nefarious activity as
is your sending a payload to the SQL Server that is in essence an
attempt to overflow the buffer of 96 bytes. That a null causes a fail
to exploit is irrelevant.

3) It is not a false positive, it is a positive match on shenanigans
targeted at the SQL server. At most it is an unsuccessful attack to
exploit the system.

> There are some signatures detecting the return address "0x42b0c9dc", but
> there are three different return addresses in the same DLL.
>
> FACT!!!
>
> Some of IDS/IPS uses "complex" ways to detect the shellcode, just like
> SNORT, but... Guess what? There are portions of the shellcode, even using
> "polymorphic shellcoding", that are still static. And due to that it is
> possible to detect the encoder instead of the rest of encrypted shellcode.
> But, guess what, again? You can do your own encoder to defeat this
> detection.
>
> FACT!!!

If you detect the attempted exploitation of the vulnerability, the
shell code used does not matter.

> > This is an easy target that recently got updated in the _signature_
> > packs of one vendor that claims to be other than a fast regex engine.
>
> Regex is just a complex way to write signatures to prevent character
> insertion in the pattern, but it still is a PATTERN MATCHING. Do you
> remember the (in)famous "CWD ~ROOT" and "CWD\t~ROOT"?

and all of the other variants. It is a more powerful pattern match but
a pattern match none the less. This is the body of my statement
"claims to be other than a fast regex engine"

> > > I think the best approach is when vendors get the knowledge of how the
> > vulnerabilties work, istead of how the exploits exploit the vulnerability.
> > This is so reactive that any new exploit / worm variant will require a new
> > signature.
> > > Keep in mind that when you know how the vulnerability can be exploited
> > is better than know how so many exploits works, but it is not that easy! A
> > signature database based on pattern is easier but gives you worng sense of
> > protection, and this worse than no protection.
> > > That said, IMHO, anomaly detection + signature database based on
> > vulnerabilties + behavior detection + any other approach other than
> > pattern means BETTER SECURITY / PROTECTION.
> > > My 2 cents.
> >
> > Do not underestimate the value of signatures in protocol modeling.
> > Especially when it is backed by a rules language that allows for full
> > protocol state tracking and analysis.
>
> Protocol modeling must understand and decode the protocol, and it just works
> if you have good and sanitized signatures, otherwise you still have and will
> have problems...

We agree and to state more specifically the issue, to effectively
model a protocol relating to exploiting vulnerabilities, the writer
must understand the different entry points and code paths that
exploitation of a vulnerability can follow. The protocol itself is
largely irrelevant at the point that code execution becomes possible,
except as a constraint on what you are allowed to do up to the point
that you get that code execution.

You can certainly model the protocol and use exception handling logic
to generically detect things that do not comply, this is where the
false positive might play, standard protocols and RFCs being optional
for some companies and all. Protocol analysis and validation in
general falls into the anomaly category and is another thing rules are
often used for.

> > All protocols have patterns, these patterns are often identifiable by
> > signatures. This is perhaps why people often confuse the snort rules
> > language with a purely signature based engine. That perception could
> > not be farther from the truth. In a simple form there are a few
> > components that come together to facilitate far more sophisticated
> > analysis.
> >
> > 1) Preprocessors - These are best described as protocol normalizers,
> > generally employed when protocols have multiple representations that
> > need to be accounted for. They can also be used for crypto,
> > behavioral, and anomaly based detection in a faster way than rules can
> > support alone.
> >
> > 2) Signatures - These are your standard pattern matching methods
> > generally employed
> >
> > 3) Rules - These allow you to do the really cool things with traffic
> > analysis without writing code.
> >
> > EG:
> >   - Find the field marker for buffer size denoted by the pattern |03 04 06
> > FE|
> >   - At the end of the marker, take four bytes and interpret them as an int
> >   - Jump that many bytes forward in the payload
> >   - From that position find the op code for open file
> >   - At the open file op code location test the value of file size, if
> > it is greater than 128 there is a vulnerable condition.
>
> Ask your preferred pen-tester how easy is to compromise this detection, even
> using either false negative or false positives.

We clearly differ on the definition of false positive, the definition
of evade (false negative), and that in the contrived case above that
you could evade it at all for the Snort engine.

We agree that there is no perfect technology and we agree that not all
signatures or engines are created equal.

------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it 
with real-world attacks from CORE IMPACT.
Go to http://www.coresecurity.com/index.php5?module=Form&action=impact&campaign=intro_sfw 
to learn more.
------------------------------------------------------------------------