Re: newbie questions

[bob_walder () mac com]

Thank you Stephen - a voice of sanity amongst those who seem a little too hung up on the "my signature database is 
bigger than yours" argument

Bob Walder
The NSS Group







On 11/1/05 3:54 pm, "Scruggs Stephen D SSgt AFWA/SCHS" <stephen.scruggs () afwa af mil> wrote:

> Mr. Paquette,
>
> I would have to concur with you about the ability to push packets down the
> wire. The first thing we (as customers) look for when researching new
> technology is how it is going to impact our current operations. Even if the
> device has the latest and greatest features and would increase our security
> policy tenfold if we used it, if there was the slightest chance it would
> drop data, we would throw it out immediately.
>
> v/r
>
>
> //SIGNED//
> Stephen D. Scruggs, SSgt, USAF
> Senior Intrusion Detection Specialist
> HQ AFWA NOSC
> 294-2463(Comm)/271-2463(DSN)
>
> -----Original Message-----
> From: Mike Paquette [mailto:paquette () toplayer com] 
> Sent: Monday, January 10, 2005 5:54 PM
> To: focus-ids () securityfocus com
> Subject: Re: newbie quetsions
>
> In-Reply-To: <41DD51DF.9080407 () immunitysec com>
>
> Dave,
>
> I've been following this thread for the last week or so.  As you may know,
> we like your CRI tool, and indeed we use it to test RPC handling and
> inspection in Top Layer's network IPS products.  I also appreciate many of
> your comments regarding the importance of the IDS/IPS properly handling IP
> fragments, TCP segments, and RPC fragments in order to defeat evasion
> attempts.
>
> I'm not quite sure, however, why you're bashing the NSS IPS tests.  Your
> comments seem to be applying a very narrowly defined criterion as the basis
> for dismissing the entire NSS IPS test suite.  Specifically, I must take
> exception to your claim that "They largely test for things you don't care
> about, such as pushing packets down a wire."
>
> As a vendor of IPS products, I can tell you that organizations planning to
> deploy network IPS technology are VERY interested in how well the IPS can
> push packets down the wire!  They all run businesses, and the packets being
> "pushed down their wire" are their lifeblood: payment transfers, sports
> bets, media delivery, internal application requests, etc.  In my experience,
> the ability of the IPS to handle legitimate traffic as a "good networking
> device" is often used as the *first* set of criteria in selecting an inline
> IPS product.  I've had many a customer who literally said that they didn't
> even want to *talk* about the protection mechanisms until we'd proven that
> our device could operate as a "good network citizen."
>
> We've run our products through the NSS IPS tests, and I just can't agree
> with the rest of your comments:
>
> > "They're not open tests."
>
> The NSS test methodologies are published in full.
>
> > "They're outdated."
>  
> The first IPS test was a year ago and the NSS methodology was brand new.
> You're right that it's mostly the same this year, save for some new
> exploits, but I would not consider it outdated.  I don't know of a more
> recent or more comprehensive set of tests for a network IPS.
>
> > "They largely test for things you don't care about, such as pushing packets
> down a wire..."
>  
> My experience shows that organizations DO care about the things that NSS
> tests for: signature coverage, baseline performance, performance under load,
> latency, application response times, anti-evasion capabilities, stateful
> operation, management and configuration.  I already  mentioned my view about
> "pushing packets down the wire."
> Bob Walder from NSS can chime in here, but my understanding is that the NSS
> signature coverage tests include many RPC-related exploits and their
> variants, run both "in the clear" and with various evasion techniques,
> including modified exploit code and RPC fragmentation.
>
> > "No scientific test should be non-repeatable"
>
> We've been able to repeat the majority of the NSS tests consistently in our
> lab.  You might be talking about the fact that the capture files for the
> attack recognition tests are not publicized.  This topic was addressed in
> the thread regarding the Tipping Point Tomahawk tool already.  Clearly the
> set of "attacks" used is the result of work that NSS has performed, and I
> understand their desire to keep that proprietary.
>
> > "and no scientific test should require such large amounts of money to
> change hands."
>
> Do you mean the test fees? The report fees? If so, why not? It's called
> "business."  Only by charging money can a test house spend the amount of
> time necessary to REALLY test advanced products like network IPS.  In fact,
> you might be able to use CRI to create your own mini-test, and charge IPS
> vendors to participate in it!  Or why not work with Walder directly to have
> him use CRI to enhance his evasion section of his test?
>
> Mike Paquette
> VP Technology,
> Top Layer Networks, Inc.
>
> --------------------------------------------------------------------------
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from CORE
> IMPACT.
> Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> to learn more.
> --------------------------------------------------------------------------
>
> --------------------------------------------------------------------------
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from 
> CORE IMPACT.
> Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
> to learn more.
> --------------------------------------------------------------------------


--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from 
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
--------------------------------------------------------------------------