Re: Specification-based Anomaly Detection

[Adam Powers <apowers () lancope com>]

> That said, imho, the best statistical
> anomaly detection on the market is still a human brain and a little
> quality time with tcpdump :). The price is right too :-).

This is absolutely the best approach. But what I'll ask is "why" the analyst
was inclined to seek the help of tcpdump in the first place? Unless there
have been "quantum leaps" in the realm of human precognition, the admin had
1) spidey-sense 2) a complaint/notification from a user or 3) an alarm/alert
from a deployed network monitoring technology. My money's one #2 or #3.
Without one of these items, the price is *NOT* right as tcpdump-style
analysis is far too inefficient in a large environment.

Behavior-based anomaly detection systems are not designed to replace the
analyst nor do his/her job for them. Like any professional's tool (a
scalpel, a type-writer, a violin) anomaly detection systems are designed to
make the operator faster and more effective at what they do. I would extend
this same functional criteria to most security or network monitoring
technologies. If it doesn't make me smarter or more effective, I don't need
it.

> With all deferences to Stefano and his recearch in the area, I haven't seen
> any of the statistical anomaly methods produce any significant results yet.
> Most sysadmins don't have much of an idea of what constitutes "normal"
> traffic patterns on their nets and I have yet to see a formal mechanical
> model that can do even less than that.

Why do you need a documented "formal model" to declare a technology
valuable? Do you think if we (Lancope) had a "formal model" we would
announce it to the world?! We're a *for profit* company. ;) Why do people
not ask ISS, Intruvert, and others to document their signatures in a "formal
model"?

Just because you, Dragos Ruiu, haven't seen "significant results" doesn't
mean they don't exist. I think you (like many others) are not quite certain
what kind of results "this kind of stuff" should produce. Therefore you
picture a utopian product with capabilities that seem incomprehensible to
the "with clue" technical individual. The truth is that these systems
probably do something in the middle of an idealistic vision of an anomaly
detection system and what's been achieved in the past. This confusion is not
your fault, but rather the vendor's for creating all this marketing "FM"
crappolla.


On 1/23/05 7:05 PM, "Dragos Ruiu" <dr () kyx net> wrote:

> On January 19, 2005 08:43 pm, Adam Powers wrote:
> > I tend to agree that claming something as "ground breaking" or
> > "revolutionary" is irritating beyond all belief.
>
> Quantuum leaps in the eye of a marketing person are rarely the paradigm
> shifts (:-) they believe they are.
>
> With all deferences to Stefano and his recearch in the area, I haven't seen
> any of the statistical anomaly methods produce any significant results yet.
> Most sysadmins don't have much of an idea of what constitutes "normal"
> traffic patterns on their nets and I have yet to see a formal mechanical
> model that can do even less than that. That said, imho, the best statistical
> anomaly detection on the market is still a human brain and a little
> quality time with tcpdump :). The price is right too :-).
>
> Big claims are easy, but I'd like to see the vendors of this kind of stuff
> back their hyperbole with some case studies outlining significant wins/kills,
> before I get excited about any of these systems...
>
> Once the merit is proven, _then_ we can start to look into the second
> order stuff like training and noise attacks...
>
> cheers,
> --dr


-- 

Adam  Powers
Senior Security Engineer
Advanced  Technology Group
c. 678.725.1028
o. 770.225.6521
f. 770.225.6501
e. apowers () lancope com
AOL IM:  adampowers22

StealthWatch by Lancope - Security  through network intelligence



--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------