RE: interesting paper on testing sig-based IDS

["Kyle Quest" <Kyle.Quest () networkengines com>]

It's interesting how there's a disconnect between
the academic research and the real world (I'd like
to note that I'm not the first one to bring it up;
similar statements have been made about academic
research when it comes to IDS design). It

The main concept in this research paper is very
much a lot like what Canvas does when it comes
to evasion (the actual evasion techniques are not
the same though). It does seem like there's a
(partial) overlap with Whisker too. That's not
the main reason I decided to write this email.

The problem that I have with the paper personally
is in section 5.4, Results (the very end of it).
They talk about how their automated approach
is superior to the manual approached used 
by NSS. I'm not sure how much the authors
of this research paper know about the tests
performed by NSS, so I can speak only for myself
(I had to deal with the tests directly 
when I worked for my old company, 
Top Layer; they were IPS tests, 
but the evasion tests are very similar). First
of all, it's not a manual effort. Second,
the actual evasion tests performed by NSS
are very much different then what the paper
describes, so the whole "our evasion tool 
is so great even though it used the same evasion
tests" just doesn't fly. 

I want to mention that I don't work for NSS 
and I'm not trying to defend them in any way.
They don't test every single exploit in the world.
They don't test every possible evasion combination.
That's not their goal. I would refer you to Bob Walder
for that... He can better explain what they do at NSS.

Also, trying to use this "mutation approach"
to generate false positives is somewhat flawed.
Sure you might be able to generate alerts on
traffic that doesn't have actual exploits,
but it won't be very useful when to comes
to real world legitimate traffic 
generating false positives.

Kyle

-----Original Message-----
From: Giovanni Vigna [mailto:vigna () cs ucsb edu]
Sent: Monday, February 28, 2005 6:45 PM
To: Kohlenberg, Toby
Cc: <focus-ids () lists securityfocus com>
Subject: Re: interesting paper on testing sig-based IDS


Well,
        We sort of suck in terms of publicizing our research.

Our tool, which is called 'Sploit', is more similar to CANVAS than to  
any other.
I haven't seen/tried CANVAS so I am not sure, but the basic ideas seem
similar.

You can get the details from the paper, but the idea is
to compose exploit templates and mutant operators.
The mutation engine applies one or more mutant operators to
an exploit template to obtain a mutant exploit. Then the exploit is run
against a vulnerable application and an oracle determines if
the attack was successful (this is necessary because even though
the mutant operator are supposed to preserve the semantics of the
exploit, things can actually go wrong in unexpected ways).
The outcome of the oracle is automatically cross-correlated
with the outputs of one or more intrusion detection systems.
By "exploring" the mutation space it is possible to find the right
composition of mutant operators to evade an IDS.
In out paper we show that using our tool we were able to
evade 9 out of 10 attacks, in the case of ISS RealSecure.

We are not distributing our code at the moment.

Best regards,

        Giovanni


On Feb 25, 2005, at 9:01 PM, Kohlenberg, Toby wrote:

> http://www.cs.ucsb.edu/~vigna/pub/ 
> 2004_vigna_robertson_balzarotti_CCS04.
> pdf
>
> It seems very similar (at least at first glance) what what's been
> implemented by
> RFP in Whisker (the anti-IDS techniques) or in Metasploit (IDS  
> confusion
> techniques).
>
> Have any/many of you seen this before? It seems like it's something we
> would have
> seen cross this list but I don't remember it doing so.
>
> t
>
> Toby Kohlenberg, CISSP, GCIH, GCIA
> Senior Information Security Analyst
> Applied Security Technology Team
> Intel Corporate Information Security
> 503-712-8588  Office & Voicemail
> 877-497-1696  Pager
> "Just because you're paranoid, doesn't mean they're not after you."
>
> PGP Fingerprint:
> 92E2 E2FC BB8B 98CD 88FA  01A1 6E09 B5BA 9E84 9E70
>
>
>
> ----------------------------------------------------------------------- 
> ---
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from
> CORE IMPACT.
> Go to  
> http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> to learn more.
> ----------------------------------------------------------------------- 
> ---


--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from 
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
--------------------------------------------------------------------------


--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
to learn more.
--------------------------------------------------------------------------