RE: Protocol Anomaly Detection IDS

["Graham, Robert (ISS Atlanta)" <rgraham () iss net>]

As you have rightly guessed, marketing buzzwards mean whatever you want
them to mean.

One way of doing IDS is to read the RFCs and trigger an event whenever
the network traffic doesn't conform to it. I first did something like
the back in 1992 -- but more as a debug/interoperability tool in order
to figure out why two applications couldn't talk to each other.

I call this "protocol-validation"; I don't do it much because, as you've
noticed, vendors suck at reading RFCs. If I triggered on every little
protocol violation, then I'd drown in a torrent of events. This is
indeed what customers are telling me when they try out competing
products that are based mostly on "protocol-validation" in real-world
networks.

One of the (marketing) advantages of protocol-validation is that you get
to claim 100% "accuracy". Because they follow the RFC and the network
traffic doesn't, then its the network traffic that is wrong. It's still
a false-positive in my book, and a lot of our protocol-validation
signatures have anti-signatures to filter out popular products that
produce invalid traffic.

Protocol-validation has a problem that most attacks are "legal"
protocols, or the attack is against a vuln that isn't defined in an RFC.
For example, there is no RFC for the port 1434/udp Microsoft protocol
attacked by SQL-slammer. Protocol-validation vendors claim that they can
detect attacks without dirtying their hands with security research -- I
disagree.

What ISS calls "protocol-anomaly" is somewhat different than other
vendors. We refer to an anomaly as something that is legal (as far as
the RFC is concerned), but otherwise odd. For example, a 100-character
password is legal as far as the FTP RFC is concerned, but we still
trigger on it as an anomaly. Note that RealSecure/1.0 and Wheelgroup/1.0
(now Cisco IDS) have been doing this sort of anomaly detection since
1996, and it has also been a big part of NFR and Intrusion.com. It's
also worked well as a "post-vuln pre-exploit" way of developing
signatures. Something like Snort waits until exploits are developed,
then develops signatures for that exploit. E.g. the Snort sigs for
SQL-slammer were for that worm, but could miss variants. The NFR and
RealSecure signatures would trigger on the VULN not the EXPLOIT, so they
catch the exploit. Protocol-anomaly detection in RealSecure has also
been pretty successful at catching things before even than the vuln is
known, but this doesn't happen nearly as often as the
post-vuln/pre-exploit period. Note that sometimes vuln announcements are
imcomplete; I've gotten burned twice in reviews because hackers found an
alternate way of exploiting something, and if I had just done a
signature-match I would have caught the variant -- but this is very
rare.

Note that none of the reviewers test the post-vuln/pre-exploit issue.
They just run known exploits. This irritates me a bit.

In answer to your original question, all vendors support some sort of
protocol-anomaly detection. Even Sort has some signatures along those
lines. Protocol-anomaly (or validation) is the preferred way that most
vendors now do signatures. I think we have more anomaly sigs and more
application protocols supported than anybody else, but then, everyone
has their spin.

BTW, Lancope does statistical anomaly, not protocol anomaly. That's a
different beast altogether.




-----Original Message-----
From: Michael L. Artz [mailto:dragon () october29 net]
Sent: Tuesday, February 04, 2003 11:07 PM
To: focus-ids () securityfocus com
Subject: Protocol Anomaly Detection IDS


I am trying to supplement our existing signature based IDS (Snort, gotta

love open source) with a protocol anomaly based one in a fairly large 
enterprise network.  I am in the fairly early stages of research, so I 
guess that the first question would be, is it worth it?

I hear the anomaly detection buzzword thrown around a lot these days, 
and can't quite get past all the marketing hype.  From what I can tell, 
protocol anomaly detection seems to be the more promising than the 
statistical for detecting new or IDS-cloaked attacks.  However the 
notion of "conforming to RFCs" leaves a lot of leeway for the vendors to

play with.  How well do these types of systems actually work?

Does anyone have any recommendations as to which systems to look 
into/stay away from?  Below is a list of some of the ones that looked 
like they might support protocol anomaly detection from their marketing 
hype, let me know if I left any out/incorrectly added any:

Lancope Stealthwatch
Tipping Point/UnityOne
ISS RealSecure Guard
Cisco IDS 4250
CA/eTrust IDS
Intruvert Intrushield
NFR Network Intrusion Detection System
Netscreen/Onesecure IDP
Symantec ManHunt

Any clues or headstarts to get me pointed in the right direction would 
be great.

Thanks
-Mike