RE: Specification-based Anomaly Detection

["Ofer Shezaf" <Ofer.Shezaf () breach com>]

Hi Thomas & Stefano,

I agree that anomaly detection is a new-comer to IDS, and in many cases
not a mature technology. But I think that due to the inherent
shortcomings of signatures, it has to be considered seriously.

As one of you mentioned, the main disadvantage of signatures is zero day
attacks.  As I see it, the significance of zero day attacks is way
underrated. Zero day attacks usually refer to abusing of vulnerabilities
before a patch or a signature has been issued, but there are those
"perpetual" zero day attacks - the bugs in the software of a specific
web site. 

The recent "phpInclude" worm is a very good example of exploitation of
such "perpetual" zero day attacks. The worm itself can be detected by
signatures as, being a publicly available code, it includes some
repeating patterns. On the other hand the same the same techniques can
be (and probably are) used by "none worm" crawlers or even manually to
attack specific sites, and are not be detected by signatures.

As to anomaly detection: I come from a company that does anomaly
detection and I feel that it is one of the ways to solve the problem
presented above. 
This might be a different perspective than yours as I believe that both
of you come from network anomaly analysis background. There are at least
two main differences between the network layer and the application layer
regarding anomaly detection:

1. The application layer carries a lot more information (after all we
analyze the entire connection and not just the communication headers).
Naturally this presents more potential for statistical modeling.

2. On the network layer, network profiling analyzes the normal behavior
of users (i.e traffic), while in the application layer we also profile
the normal behavior of the application.

Saying that, anomaly itself usually identifies that something is wrong
but not what is wrong. We use two important additional mechanisms to
derive actionable information:

1. Application Layer Signatures - these signatures detect content that
may indicate an application layer attack. These signatures are much more
prone to false positives and may be more computationally complex to
detect. Simple examples are the word "select" (used in SQL injection)
and Win 32 assembly code (buffer overflows). Application signatures are
effective to determine an actionable item once an anomaly was detected.

2. Correlation - another important aspect of application layer attacks
is that they are not encapsulated in a single packet. Correlation
enables us to both correlated different anomalies to generate more
meaningful events and to follow longer term attacks.

~ Ofer

Ofer Shezaf
CTO, Breach Security

Tel: +972.9.956.0036 ext.212
Cell: +972.54.443.1119
ofers () breach com
http://www.breach.com 


> -----Original Message-----
> From: Stefano Zanero [mailto:zanero () elet polimi it]
> Sent: Friday, January 07, 2005 6:06 PM
> Cc: focus-ids () lists securityfocus com
> Subject: Re: Specification-based Anomaly Detection
>
> Thomas Ptacek wrote:
>
> > What makes you think that information about supposed RFC violations
on
> > your network will be actionable?
>
> This is an extremely good question: is anomaly detection of any sort
> trustable enough for intrusion prevention purpose ?
>
> > Most people don't find information
> > about supposed malicious traffic to be genuinely actionable.
>
> Or informative. Unless you have a very specific packet trace and a Tom
> Ptacek-like guy to read it :)
>
> > I'm not
> > aware of any evidence, not even anecdotal, of new vulnerabilities
being
> > discovered by anomaly detection systems of any stripe.
>
> Here I disagree with Tom. I'd say that anomaly detection systems are
not
> widely deployed in the wild, so we have no data on their ability to
> strengthen corporate defenses. The only widely tested anomaly
detection
> tools are statistical, rate-base... in which case I certainly agree
with
> Tom.
>
> I also agree with Tom that there's still a long road ahead before
having
> good anomaly detectors. The fact that two people involved in
researching
> (me) and selling (Tom :-))) anomaly-based technologies are so careful
in
> what we think our beloved creations can do, should warn you.
>
> This is not technology ready for prime time.
>
> > Replacing signature IDS is not one of those things.
>
> Absolutely, what would be the use ?
>
> Best,
> Stefano Zanero
>
> Politecnico di Milano - Dip. Elettronica e Informazione
> www.elet.polimi.it/upload/zanero
------------------------------------------------------------------------
--
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from
> CORE IMPACT.
> Go to
http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> to learn more.
------------------------------------------------------------------------
--


--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------