Re: Metasploit and efficacy of IPS devices

[Jason <security () brvenik com>]

H D Moore wrote:
> Disclaimer: I work on both the Metasploit and BreakingPoint products.
>
> Many test labs use Metasploit (along with Core, Canvas, and commercial 
> tools like BreakingPoint) to perform IPS certification. While this isn't 
> a 1:1 of what IPS detects what exploit, the public reports are a good 
> start. 
>
> http://nsslabs.com/content/category/4/22/42/
> http://www.icsalabs.com/icsa/topic.php?tid=4d56$eed283d1-c66d427c$af04-d91faa8c
>
> Three things to keep in mind when evaluating an IPS for attack detection.
>
> 1) Most IPS products ship with a limited number of signatures enabled by 
> default. In some cases, this signature set is less than a third of the 
> total, resulting in terrible out-of-the-box coverage. If you are going to 
> perform your own test, I suggest running it once with out-of-the-box 
> settings, and again with all signatures enabled.

To add to the statement.

Many IPS systems will generate side channel events and possibly the
exploitation itself. By enabling all signatures you make it a
requirement to measure every attack and the events it generates
individually. If you do not instrument for testing each attack and then
measure it you will be easily misled.

EG: Small fragments have nothing to do with the actual attack, just a
potential evasion method, yet you will get events and possibly block it.
Change the fragment size to be one byte over the defined threshold and
you may well have a successful attack and evasion.

> 2) The better exploit tools (Metasploit and BreakingPoint for sure) allow 
> you to change how attacks are delivered via user-configurable options. In 
> Metasploit 3, the "show evasions" command will list numerous parameters 
> that change how traffic is generated and delivered. In the BreakingPoint 
> case, each Strike supports a wide range of evasion options, from Layer 3 
> all the way to the exploit parameters and payloads. 
>
> 3) When testing with exploits designed to give you a shell (Metasploit, 
> Core, CANVAS, etc), the IPS may detect the attack based on exploit and 
> tool-specific traffic patterns. For example, one IPS detects common 
> shellcode decoder stubs in the default signature set. This can scew your 
> test results, since these decoder stubs are trivial to modify.

More side channel stuff...

I've seen many fall prey to thinking the attack was blocked simply
because they did not get the shell.

> Hopefully that wasn't too spammy ;-)

Never!

------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it 
with real-world attacks from CORE IMPACT.
Go to http://www.coresecurity.com/index.php5?module=Form&action=impact&campaign=intro_sfw 
to learn more.
------------------------------------------------------------------------