IDS mailing list archives
Re: newbie questions
From: bob_walder () mac com
Date: Fri, 14 Jan 2005 10:34:19 +0100
I will say the same thing to you I sad to Dave Aitel - you need to read our methodology more carefully. The exploits to which you refer by name are our BASELINE exploits - they are chosen to ensure that EVERYONE has the signature in their database - there are about 7 of those ONLY - they are used to do some of the basic evasion stuff. Yes, they are old - yes they are simple - that doesn't matter because they are not part of the DETECTION test. If you see an explanatory reference such as (i.e. change GOBBLES to GOOBLES") then it is just that - an explanation of the TYPE of evasion we are doing there - it is NOT the extent of the testing. Why would we tell everyone (including the vendors who will be in future tests) EXACTLY what we are going to do in our evasion tests? We just provide EXAMPLES. UNDERSTAND the methodology before making stupid claims. You don't KNOW what is in our attack library because that is the one part of the methodology we do not detail. People who get too hung up on this part of the test don't understand about the realities of IPS devices. You are focussing ONLY on the detection and nothing else when you dismiss the rest of our test suite so blithely. As it happens, we have some VERY recent stuff in our library and it is updated/changed for every edition of the report We DO, however, understand that IPS is not IDS - I am not sure you do.... Come up with better tools than fragroute and Whisker/Nikto for testing those PARTICULAR evasion techniques and we will gladly use them. Once again you seem to imply that because we say we use "Whisker evasion techniques" in our methodology that we are "outdated" because we should be using Nikto. Once again, you do not understand what we are testing. We do NOT use Nikto/Whisker as a Web scanner - we have better means to test whether IPS can block stuff like that. We say quite clearly that we use the Whisker EVASION TECHNIQUES (on exploits of our own, not those in the standard test databases you get with these tools). Nikto is based on libwhisker - the same techniques apply. Once again you have focussed on a very small point ("Oh my GOD! They don't use Nikto!") without attempting to understand what exactly it is we are testing. FYI We STILL find devices that cannot do TCP seg reassembly properly - and in the latest test, one device even failed two of the Whisker test cases. Just because those tools have been around for a while does not make them irrelevant - if we did NOT test against such commonly available tools, we would be doing our readers - and the vendors - a disservice. FYI - we do far more than JUST fragroute/whisker.... Contrary to what some other misinformed people have claimed, we even... Horror of horrors.... Do MSRPC evasion testing.... Yes, really! I have no problems with people making constructive criticism, but you and Dave Aitel are quite simply playing stupid public guessing games and getting it wrong every time You need to stop making ridiculous claims and get your facts straight before posting in public forums. I am always happy to answer questions about our methodologies. Bob Walder The NSS Group On 12/1/05 7:02 pm, "Julius Detritus" <julius.detritus () ifrance com> wrote:
About NSS tests:"They're not open tests."The NSS test methodologies are published in full.You don't have the details of the tests (not even the "baseline" signatures)."They're outdated."The first IPS test was a year ago and the NSS methodology was brand new. You're right that it's mostly the same this year, save for some new exploits, but I would not consider it outdated. I don't know of a more recent or more comprehensive set of tests for a network IPS.They are outdated. The most recent exploit tested must be two years old... They are copy and paste from IDS tests which are far older. And the whole methodology is not appropriate. IPS are not IDS. For IDS "false alarms" generated by out of session packets (like the one snot would raise on snort) are not acceptable as it would confuse administrators in their research for effective attacks. In the case of IPS it is different. OK, it was not a real attack but who cares. The purpose of IPS is to block. Who cares if it blocked attacks out of session? It was not legitimate anyway. But to understand that, you need to understand IPS, and to be used to security operations (devices management, post-mortem audits, forensics analysis and the like...)"They largely test for things you don't care about, such as pushingpackets down a wire..."My experience shows that organizations DO care about the things that NSS tests for: signature coverage, baseline performance, performance under load, latency, application response times, anti-evasion capabilities, stateful operation, management and configuration. I already mentioned my view about "pushing packets down the wire."Do you really care about the phf exploit? Or maybe the old sshutupteo from gobbles? Are you talking about organizations or museums?Bob Walder from NSS can chime in here, but my understanding is that the NSS signature coverage tests include many RPC-related exploits and their variants, run both "in the clear" and with various evasion techniques, including modified exploit code and RPC fragmentation.Anti-evasion is Whisker (not nikto, I said whisker) and fragroute 1.2... Modified exploits are common ones with strings changed (GOBBLES to GOBBLED)."No scientific test should be non-repeatable"We've been able to repeat the majority of the NSS tests consistently in our lab.So your exploit database must be very old My 0.02$ Julius _____________________________________________________________________ Envie de discuter gratuitement avec vos amis ? Téléchargez Yahoo! Messenger http://yahoo.ifrance.com -------------------------------------------------------------------------- Test Your IDS Is your IDS deployed correctly? Find out quickly and easily by testing it with real-world attacks from CORE IMPACT. Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 to learn more. --------------------------------------------------------------------------
-------------------------------------------------------------------------- Test Your IDS Is your IDS deployed correctly? Find out quickly and easily by testing it with real-world attacks from CORE IMPACT. Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 to learn more. --------------------------------------------------------------------------
Current thread:
- Re: newbie questions bob_walder (Jan 17)
- <Possible follow-ups>
- Re: newbie questions bob_walder (Jan 17)