RE: 10Gbps IPS - what you need to know

["Vikram Phatak" <vphatak () nsslabs com>]

Hi Ravi,

We've been quite busy - hence the delayed response.  I wanted to put your concerns to rest regarding NSS testing 
methods.  All of our performance tests are run in the context of security effectiveness.  If a device begins to leak 
attacks under load, it is an automatic FAIL.

We test this by having the vendor install and configure their system as they would on a customer site.  We prefer the 
vendor use a "default" profile, or something of that nature that their customers would likely use - if tuning is 
required, we highlight that in our report.   

Step #1: We then test the effectiveness of the device: How good is it at stopping exploits?
Step #2: Next, we run performance tests WITHOUT changing the security settings.
Step #3 Once the maximum performance numbers have been established, we re-run the performance tests at 25%, 50%, 75%, 
and 100% max load for the device - injecting the same exploits that were CAUGHT in Step #1.  This ensures that security 
effectiveness is NOT being throttled back as performance increases.

Our methodologies are quite specific in this regard and we follow them religiously.

Also, I would point out that there have not been many IPS Products recently certified by NSS.  Look to the product 
version you are running and see if it is on our certified list?  

We wrote a few blog entries on the topic late last year.  
http://nsslabs.blogspot.com/2008/10/why-doesnt-nss-labs-have-report-on.html and 
http://nsslabs.blogspot.com/2008/10/how-long-is-product-certification-valid.html 


Hopefully this answered your question?

As to other comments/questions in this thread:  I will respond shortly to clarify.

Best,

  -Vik


Vikram Phatak
NSS Labs



-----Original Message-----
From: listbounce () securityfocus com [mailto:listbounce () securityfocus com] On Behalf Of Ravi Chunduru
Sent: Wednesday, March 04, 2009 11:48 PM
To: Trygve Aasheim
Cc: c-info () blaisnet com; focus-ids () securityfocus com
Subject: Re: 10Gbps IPS - what you need to know

I was hoping  either NSS or vendors who had gone through the
certification to say  that all sessions/packets go through the IPS
detection in all cases as part of performance measurement. There is
absolute silence on my previous email.

Silence is enforcing the points made in earlier email that IPS devices
skip Intrusion analysis upon very small load on the system.  I was
hoping that somebody is going to speak out and prove otherwise.

Ravi

On Sat, Feb 28, 2009 at 4:15 PM, Ravi Chunduru
<ravi.is.chunduru () gmail com> wrote:
> Hi,
>
> This concerns many people I am sure.   I hope certification agencies
> are reporting performance numbers where all packets are going through
> complete inspection. If what you said is happening, then certification
> agencies would lose their credibility.
>
> Any comments from NSS?
>
> Thanks
> Ravi
>
> On Fri, Feb 27, 2009 at 9:18 AM, Trygve Aasheim <trygve () pogostick net> wrote:
> > Sure.
> >
> > But to clearify; it didn't bring throughput down to 15mbit, but we where
> > seeing signatures/filters that started to fail when the traffic was at about
> > 15mbit, with less that 2000 sessions.
> >
> > Now this isn't unusual with an IDS solution that uses pure regex signatures.
> > But some IPS vendors claim that their solutions ain't running anything near
> > regex, and their performance are way up there...sure.
> >
> > Many IPS solution has a max latency threshold. So if the IPS uses more than
> > X amount of ms before it has analyzed the traffic, the traffic is passed
> > through. This is so that the IPS doesn't become a bottleneck under heavy
> > load.
> >
> > But if you have (which many companies does) different service segments (like
> > one segment with databases, one with webserver etc) you might end up with an
> > IPS where a policy for one segment is a lot of different http, https, sql
> > injection, xss signatures/filters that all try to analyze the same traffic.
> > Add to these the filters for protecting apache, iis, SunOne and so on, and
> > the different version on different operating systems - and you end up with
> > one massiv policy if you're in a big company.
> >
> > Then I promiss...gbit performance where all the filters that you really want
> > to run are on...fails. NSSlabs are more than welcome to ask for more
> > detailed information of course, so that their certifications might show more
> > than just throughput with a policy on, but also maybe throughput with a
> > policy running, where the policy is actually being applied to the data at
> > gbit speed.
> >
> > The traffic pattern was customer traffic btw. Real world traffic.
> >
> > T
> >
> >
> > Ravi Chunduru skrev:
> > > > We've seen gbit certified solutions starting to fail at 15mbit with <2000
> > > > sessions during PoC's....
> > >
> > > This is really interesting. Can you throw some more light on traffic
> > > pattern which brings down the performance to 15Mbps?
> > >
> > > Ravi
> > >
> > > On Mon, Feb 23, 2009 at 9:16 AM, Trygve Aasheim <trygve () pogostick net>
> > > wrote:
> > > > Another question would be:
> > > >
> > > > - How big is the rule base?
> > > > - Any exceptions
> > > > - How many filters/signatures/detection features failed to analyze the
> > > > traffic before the latency treshold was exceeded?
> > > > - Is the rule base based on a scenario where you for example pretend to
> > > > protect a windows server and workstation network, and therefor enable all
> > > > signatures for this - and turn off all *nix signatures? Or the other way
> > > > around? Or a pure web-/app-/database server network?
> > > >
> > > > A lot of these tests fail to test the devices in a "near real world
> > > > scenario" where the IPS is configured with an adjusted rule base based on
> > > > typical assets, risks, firewall rules, exceptions, vlan tags etc.
> > > >
> > > > We've seen gbit certified solutions starting to fail at 15mbit with <2000
> > > > sessions during PoC's....
> > > >
> > > > T
> > > >
> > > > C-Info skrev:
> > > > > The question I would also ask is was this complete capture or sampling
> > > > > of
> > > > > the traffic?
> > > > >
> > > > > Curt
> > > > >
> > > > > -----Original Message-----
> > > > > From: listbounce () securityfocus com [mailto:listbounce () securityfocus com]
> > > > > On
> > > > > Behalf Of Addepalli Srini-B22160
> > > > > Sent: Thursday, February 19, 2009 1:57 PM
> > > > > To: Ravi Chunduru; rmoy () nsslabs com
> > > > > Cc: focus-ids () securityfocus com
> > > > > Subject: RE: 10Gbps IPS - what you need to know
> > > > >
> > > > >
> > > > > Copied from the test report:  "The device ably supported over 11Gbps
> > > > > of traffic with the larger HTTP response sizes (21KB) and lower
> > > > > connections per second (5,000 CPS per Gigabit of traffic) found on
> > > > > typical corporate networks".
> > > > >
> > > > > It appears to be some calcualtion mistake!  It comes to around
> > > > > 820-830Mbps (21Kbytes * 5000 ), not 11Gbps throughput!
> > > > >
> > > > > > I think you missed "5000 CPS per gigabit of traffic". Since it is 10G
> > > > >
> > > > > box, I would assume that there was 50000 CPS in total which gives around
> > > > > 8.5Gbps. If you add usual overheads TCP header, IP header, Ethernet
> > > > > header, the total throughput might go beyond 8.5Gbps.
> > > > >
> > > > > Regards
> > > > > Srini