RE: Current state of Anomaly-based Intrusion Detection

["Andrew Plato" <andrew.plato () anitian com>]

This all depends on your definition of an "anomaly" detection engine. I would say that anything that establishes 
"norms" for protocols or traffic is, in essence, an anomaly detection system. It is comparing what it sees to 
established norms. This would therefore include most of the popular IDS/IPSs on the market. Many of the commercial 
IPS/IDS establish norms for each application protocol (such as a User ID field shouldn't be more than 150 bytes). Then 
when there is a violation of that norm, its considered an "anomaly." 

One of the painful lessons I have learned in the past 6 years of installing and tuning IPS and IDS is that most 
networks are a hodge podge of "anomalies."  Averaging traffic doesn't work because businesses often have spikes and 
valleys in traffic. Detecting protocol anomalies is a mess as many applications do not use network and application 
protocols properly. SNMP comes to mind. This is a protocol that is grotesquely misused. 

As such, there really is no perfect system. People on all sides will scream and howl how their system is superior and 
can see through walls and bend space-time to detect the hackers. The reality is, most of the IPS/IDS on the market have 
a variety of weaknesses. 

Pure signature-based IPS/IDS may be making a comeback. This is partially because the hardware has now progressed to a 
point where signature-based systems can process packets and streams quickly enough not to cause significant latency. I 
have been spending a lot of time with TippingPoint's UnityOne product these days and their engine is interesting in 
that way. While it can do a lot of protocol anomaly checking, it does not bother alerting on those (by default.) They 
have speeded up the signature engine to the point where they can pick out known bad stuff and limit the protocol 
anomaly noise. This dramatically limits their false positive rate. 

Network intrusion detection isn't really that new. Its been around for a decade or more. Anomaly detection isn't really 
new either. Its been around for 5-6 years. I think what is new is the expectations for IPS/IDS. The technology has 
evolved to a point where being able to detect every weird variance in a network might not be as useful as originally 
thought. 

Andrew Plato, CISSP
President 
Anitian Enterprise Security
www.anitian.com 



 

-----Original Message-----
From: Göran Sandahl [mailto:goran () gsandahl net] 
Sent: Friday, February 25, 2005 4:05 PM
To: focus-ids () lists securityfocus com
Subject: Current state of Anomaly-based Intrusion Detection

Hi all.

I'm trying to get a picture of the current state of Anomaly-based network-monitoring-systems. In other words, 
Anomaly-based IDSs (are they really called ADSs?). 
After following the thread "Specification-based Anomaly Detection", I've realised that this question probably has allot 
of answers. However, I'd be very glad if you would take the time and write a couple of lines on what you think of the 
techniques that are used today, and whats needed for the future. 
I'm interested in "both sides of the story", so please tense your muscles and raise your voice ;)

As i figured, there are two different techniques that these systems work upon. 
Either, they are based on specifications (for example, hardcoded 200kb/s SMTP-traffic is normal) , or on statistic 
(Based on an "average". For example, 20 current TCP-sessions is normal). How does these techniques really work? How are 
they implemented today? How is this statistical information usually gathered? 

Also, signature-based IDSs are vulnerable to false alerts of different kinds (postitive, negative etc). I can imagine 
that anomaly-based techniques might suffer even worse to this. True?

And finally, while "reading through the lines" on some of the posts to the thread mentioned above, I got a feeling that 
this technique isn't yet ready for prime-time yet. Why is this? As I figure, the whole idea with network intrusion 
detection is pretty new, and none of the techniques seems to be without flaws.

Thanks in advance
Cheers
Göran

--

Göran Sandahl
mail:        goran () gsandahl net
web:         http://gsandahl.net

--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------




--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------