IDS mailing list archives
Re: newbie quetsions
From: avi chesla <chesla () 012 net il>
Date: 12 Jan 2005 07:50:51 -0000
In-Reply-To: <41DD51DF.9080407 () immunitysec com>
Received: (qmail 13780 invoked from network); 7 Jan 2005 00:27:04 -0000 Received: from outgoing.securityfocus.com (HELO outgoing2.securityfocus.com) (205.206.231.26) by mail.securityfocus.com with SMTP; 7 Jan 2005 00:27:04 -0000 Received: from lists.securityfocus.com (lists.securityfocus.com [205.206.231.19]) by outgoing2.securityfocus.com (Postfix) with QMQP id 110EB14375C; Thu, 6 Jan 2005 17:32:05 -0700 (MST) Mailing-List: contact focus-ids-help () securityfocus com; run by ezmlmDave Aitel wrote:
I guess the interesting thing is that you actually bought something for your millions of dollars. Or perhaps it's a look into the Speed vs. Accuracy trade off. Lots of other people have spent millions of dollars on professional engines, but still fail the simple tests like this because all nss.co.uk is testing for is extremely old attacks and whether an IDS can take the load of millions of packets at once. This is going to favor Snort-like systems largely at the expense of parsing engines. I think it's telling that nss doesn't test MSRPC at all. It's funny how the IDS industry has tuned itself. But set your MTU low enough, and you can bypass some systems even if you're the only packets on the wire. Doing SMB fragmentation basically guarantees it. If you're looking for a misleading test, the NSS.CO.UK tests are what you want. They're not open tests. They're outdated. They largely test for things you don't care about, such as pushing packets down a wire. No scientific test should be non-repeatable, and no scientific test should require such large amounts of money to change hands.
I really suggest reading the reports that NSS issues including their market overview and test methodology in order to learn about how to analyze and test security devices or any other communication devices. To say that NSS's tests are out of date is simply not true. Evaluation of IPS products raises a great challenge for the evaluator. In my experience, the NSS group does a very thorough and, perhaps most importantly, un-biased work with their round of tests of IPS devices. By examining NSS's test methodologies (published in their site and in every report they issue), it is easy to recognize the level of understanding that the NSS group has in regarding to the IPS market and product positioning (this understanding is the first step in establishing the correct test scenarios and success criteria). Regarding to Evasion techniques, NSS's tests comprise more than enough methods that try to evade detection. These include: Packet fragmentation which include 19 different methods of IP packet fragmentation and Stream segmentation, URL Obfuscation which include 9 URL obfuscation techniques (e.g., URL encoding, premature URL ending, session splicing etc), other miscellaneous evasion techniques... Of course there will always be new evasion techniques but it seems that NSS has chosen to use the most updated and common ones. Let's remember that no test can include all the possible evasion techniques but the important thing is to aim as high as possible. NSS includes also special evasion techniques in order to test rate-based NIPS which are usually based on time-dependant thresholds. In order to test these detection engines NSS generates DoS attacks, network scans and self-propagating Worm activities with different delays between packets(e.g., very slow scans, random time between events, slow TCP connection floods, slow SYN attacks etc.). In this way NSS analyzes how sophisticated these rate-based detection engines are. According to NSS reports, they have all the equipment and experience that is needed in order to simulate background traffic that emulates "real" world legitimate user behaviors (throughout several popular applications). This is a very important capability that helps to reveal false positive and misdetection percentages of the detection and prevention engines maybe the most important test for IPS devices (as high percentages of false positive renders the IPS devices useless). NSS indeed pushes the products to their limits. I think that this is certainly necessary in order to reveal how much "brain" work was invested in the hardware and software architecture. NSS's performance test includes playing with parameters such as number of simultaneous TCP connections, TCP connection rates, Packet sizes, packet rates, etc. This capability allows an analysis of the immunity of the detection engines against false positive and misdetections. It is also interesting and educating to see how NSS approaches differently rate-based NIPS and Content-based NIPS with their performances and false positive rate tests. Avi Chesla. -------------------------------------------------------------------------- Test Your IDS Is your IDS deployed correctly? Find out quickly and easily by testing it with real-world attacks from CORE IMPACT. Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 to learn more. --------------------------------------------------------------------------
Current thread:
- Re: newbie quetsions, (continued)
- Re: newbie quetsions Jason (Jan 06)
- Re: newbie quetsions Dave Aitel (Jan 06)
- Re: newbie quetsions (on how much Snort sucks) Martin Roesch (Jan 11)
- Re: newbie quetsions (on how much Snort sucks) Dave Aitel (Jan 11)
- Re: newbie quetsions (on how much Snort sucks) Martin Roesch (Jan 11)
- Re: newbie quetsions Dave Aitel (Jan 06)
- Re: newbie quetsions Jason (Jan 06)
- RE: newbie quetsions Julius Detritus (Jan 12)
- Re: newbie quetsions Rainer Duffner (Jan 17)
- About IPS testing (was: newbie quetsions) Julius Detritus (Jan 19)
- Re: About IPS testing Tod Beardsley (Jan 24)
- Re: newbie quetsions Stefano Zanero (Jan 14)