RE: Specification-based Anomaly Detection

["Ofer Shezaf" <Ofer.Shezaf () breach com>]

Hi Stefano & Toby,

I'll refrain from in-lining as it becomes cluttered down there, and I
feel that everybody is saying the same thing just with different
emphasis and a lot of arguments about wording (got to lawyers level)....
And I swear not to use "actionable" in non-commercial situations.

So I'll try to roll the discussion forward:

Stefano writes that a host and a port define a listening application and
then you carry on about detecting an application automatically. While
mail is an application, it is a static application in the sense that it
behaves very much the same over time and between users and sessions (and
so does FTP and so on).

I feel that the mind set of the discussion was about such applications,
and that an IDS system (signatures or anomaly based) for such protocols
would not be much different than a network IDS.

I would like to call the types of applications that my company's
products handle "dynamic applications". I'm referring to interactive
http based applications. Why are they different? For many reasons, only
some of them directly related IDS, but all have security relevance:

1. Only widely used system that allows a very large community to write
client server applications (hence the tons of poor coding).
2. Protocol elements are polymorphic, not just the content, and are
changed by the above "programmers".
3. Only widely used system where code is constantly downloaded by the
user.

And as a result, a lot more action...

Does this make intrusion detection in web applications deferent? Based
on our experience with out product I think so.

Why? 

Probably because the balance between know how and mathematical analysis
is different. When I think of it, our product includes a lot of implicit
know-how about http, html and how different application environment use
it. We don't have to apply abnormal behavior algorithms to a steam of
information but to clearly identified attributes of transactions that we
know quite a lot about.

In some ways this is more similar to HIDS than to NIDS (And by the way,
we also passively decrypt SSL - if we get the key - so even less
difference than a host IDS).

Another issue evolved around my assertion that the protocol is
polymorphic. When I stated in a previous e-mail that we learn the
application behavior and not the user behavior, I referred, in terms
more commonly used in IDS that we learn the protocol. As the protocol is
defined by the specific programmer at the organization building the web
site, learning it and validating that users are in conformance provides
a layer of security that I'm not sure should be called abnormal behavior
detection in the common IDS terminology.  

~ Ofer

Ofer Shezaf
CTO, Breach Security

Tel: +972.9.956.0036 ext.212
Cell: +972.54.443.1119
ofers () breach com
http://www.breach.com 


> -----Original Message-----
> From: Stefano Zanero [mailto:zanero () elet polimi it]
> Sent: Tuesday, January 11, 2005 11:29 AM
> To: Kohlenberg, Toby
> Cc: Ofer Shezaf; focus-ids () lists securityfocus com
> Subject: Re: Specification-based Anomaly Detection
>
> Kohlenberg, Toby wrote:
>
> > Stefano, could you expand on which part you agree with? I'm really
> > confused to think that you would agree that anomaly detection would
> > be new to IDS.
>
> I would agree that:
> - anomaly detection is needed as a complementary approach to misuse
> detection because of the inherent limits of the latter
> - and that anomaly detection (in particular techniques which are not
> rate-based) is a relative "newcomer" in the COMMERCIAL field of
> intrusion detection, where most of the products are built on a misuse
> detection approach.
>
>
> > > > is zero day
>
> > > Or highly polimorph attacks, yes.
>
> > Or custom-written attacks
>
> Absolutely correct !
>
> > Really? What about apps that all tunnel over a single port?
>
> That would be a problem even if you work at application layer ;)
>
> Please note that Ofer was not advocating HOST-based intrusion
detection
> but NETWORK-based approaches working at layer 7
>
> > Are you getting the application that IANA says runs on that port or
> > are you getting SAP using telnet on some random port or Cisco using
> > HTTP on yet another random port?
>
> That's something that the algorithm we have developed can recognize ;)
>
> > > This is basic misuse detection, it does not mean you can deliver an
> > > actionable anomaly detection result.
> >
> > No, but it does give you a much better chance of finding
"actionable"
> > (or ignorable)
>
> Yes,  but since we are discussing wether or not ANOMALY detection is
> "actionable" (I'm not a native speaker but this word sounds horrible
to
> me :) this objection is not relevant. Or better, it says exactly what
> Tom and I were saying: anomaly detection is not, and this is a
> disadvantage wrt misuse detection.
>
> Best,
> Stefano
------------------------------------------------------------------------
--
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from
> CORE IMPACT.
> Go to
http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> to learn more.
------------------------------------------------------------------------
--
>


--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------