RE: performance metrics for IPS systems?

[THolman () toplayer com]

> > I'm planning on demanding that the IPS systems perform at >225,000
> > packets/second (100% of packets inspected) with <.5ms latency per
> > packet.  Is this reasonable for an IPS?

I would also insist on 0% packet loss.  These are all perfectly reasonable
expectations from an IPS, however, these figures cannot be met (at
medium-to-heavy loads) with certain IDS vendors who all of a sudden are
claiming IPS ability due to market pressures.  

In fact, a lot of IPS vendors will not actually publish such performance
statistics, and it has been up to the unsuspecting buyer to find out later
on that what they bought is simply not up to scratch.

This is where NSS tests become so invaluable - Bob does great work of
blowing away the smoke and smashing the mirrors so that potential buyers can
make choices based on real technical credibility, rather than marketing
spiel !

> That being said, you probably won't find an IPS that introduces more than
> 1ms of latency.

You SHOULDN'T find an IPS that introduces this amount of latency, but it is
invariable when you subject a PCI bus to a heavy load.  This latency is also
unacceptable on core networks and many web-based applications.

My parting message - if you're in the market for an IPS, be very careful and
ask for technical proof for all the things that IPS can say it can do before
you even think about letting it onto your network !

Regards,

Tim


-----Original Message-----
From: Massimo [mailto:massimo.mail () quipo it] 
Sent: 12 February 2005 19:16
To: Mike Frantzen
Cc: p z; focus-ids () securityfocus com
Subject: Re: performance metrics for IPS systems?

We did some test with stress test equipment on the capability to handle 
hight traffic load with low latency on some "diffused" IPS.
I can tell you will have problem with some IPS product with that high 
load of packet. There are also commercial Gigabit IDS that lose traffic 
(doesn't slow, but lose) with that number of packet (225,000 packet/s 
can be close to a full gigabit with real packet size).
 
I am sorry but I have a NDA on that test and can't give you more detail.

Best Regards,
                   Massimo

On 09/01/2005 14.49, Mike Frantzen wrote:

> > I'm planning on demanding that the IPS systems perform at >225,000
> > packets/second (100% of packets inspected) with <.5ms latency per
> > packet.  Is this reasonable for an IPS?
> >    
>
> Just be careful how you measure that .5ms latency limit.  If you do a
> single ping without background traffic against an IPS that does
> interrupt polling then you'll see latency of about 1ms or 10ms
> (depending on the underlying operating system used).  That latency
> will start to drop once you have over 1000pps and will gradually
> converge towards zero.
>
> I'm not sure which IPS vendors do interrupt polling to gain performance.
> It wasn't worth it for us.
>
>  
>
> > - What is the acceptable/standard latency per packet for an IPS?
> >    
>
> Humans begin to notice latency at about the 200ms mark (call it 100ms to
> account for the return packet).  TCP behavior changes at 30-100ms unless
> the stack does round trip time measurements.  Online gamers get cranky
> at the 80-100ms mark.
>
> That being said, you probably won't find an IPS that introduces more than
> 1ms of latency.
>
> .mike
> frantzen@(nfr.com | cvs.openbsd.org | w4g.org)
> PGP:  CC A4 E2 E8 0C F8 42 F0  BC 26 85 5B 6F 9E ED 28
>
> --------------------------------------------------------------------------
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from 
> CORE IMPACT.
> Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
> to learn more.
> --------------------------------------------------------------------------
>
>
>  

--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from 
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
--------------------------------------------------------------------------

--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from 
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
--------------------------------------------------------------------------