Re: Current research on IDS

[Maarten Van Horenbeeck <maarten () daemon be>]

Hi Mark,

Interesting question. There is in fact quite a bit of research going on
in the field of intrusion detection. Perhaps interesting to know is that
there is a lot of applied work as well - at the end of August, the
Computer Security Research Centre at NIST published a draft special
publication 800-94, which serves as a guide to people considering the
use of Intrusion Detection and Prevention systems.

This draft document, while it doesn't constitute research 'pur sang',
does indicate a large degree of interest in the commercial space as to
where and how to implement these types of technical controls. You can
find it here:

http://csrc.nist.gov/publications/drafts/Draft-SP800-94.pdf

Most research on IDS focuses on taking the model a step further. Gone
are the days when regex-matching signatures were the main focus of
intrusion detection. While they still underpin most production usage
today, not all modern threats lend themselves to detection using these
fairly simple mechanisms. This is especially valid for one-off's,
attacks designed specifically to attack a certain organization, often
used in industrial espionage.

What is gradually being understood is that the art of intrusion
detection requires a solid intelligence underpinning. That's why I'm
going to refer you to a document which is in fact unrelated to IDS - it
deals with a technical solution to an "intelligence problem".
Nevertheless, as someone with a background in deploying enterprise IDS
systems, this is probably one of the more interesting papers I've read
over the last few months.

"Out of the Ordinary: Finding Hidden Threats by Analyzing Unusual
Behavior", by a number of researchers at RAND:

http://www.rand.org/pubs/monographs/MG126/index.html

The gradual realization of this move from detection to actually
benefiting from the information gathered through IDS is one of the
reasons why we're seeing so much use of SIM tools and even event
correlation on the sensor level. This branch of software now offers much
more than simple correlation and matching, it has truly become one of
supporting analysis and synthesis of security intelligence.

Actual work on improving detection is mainly focused on the use of
hidden markov models. You have an event A, influenced by (or correlated
to, by a known probability) an environment B. An object C describes the
value of event A, but nothing about environment B. Based on what you
learn from C, you do not know B but you can deduce the likelihood of its
state over time.

In the field of intrusion detection this translates to new methodologies
of profiling actions initiated by users to identify those that do not
match with the expected behavior. These usually are self-learning
systems as opposed to those that use 'rules' to detect when something
happening on a system or network is an anomaly. One of the major
challenges is for these systems to take into account base-rate
information, or information that should influence the way all learned
items is interpreted. Defining boundaries on what influences human
behavior requires a sociological perspective on intrusion detection.

Other intriguing research deals with how to combine intrusion detection
probes with other technologies to make them more effective. A
preliminary step was to combine probe output with information gathered
from vulnerability assessment systems. This allowed for a correlation on
the network between vulnerabilities and threats.

Further steps that will gradually move from the academic field onto the
market are the linking of IDS probes with honeypots to e.g.
automatically generate signatures based on probable attacks. While the
certainty of such event truly being an attack will never be absolute, a
signature (or profile, if you wish) can then automatically be
distributed across the probe base. Once an incident has been confirmed
to be an actual attack, information security analysts will be able to
identify the spread of a certain attack pattern across the organization.

The sector is about to get real interesting.

Cheers,
Maarten

-- 
Maarten Van Horenbeeck, CISSP
maarten () daemon be - http://www.daemon.be/maarten

------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it 
with real-world attacks from CORE IMPACT.
Go to http://www.coresecurity.com/index.php5?module=Form&action=impact&campaign=intro_sfw 
to learn more.
------------------------------------------------------------------------