IDS mailing list archives
RE: Intrusion Prevention requirements document
From: FinAckSyn <finacksyn () yahoo co uk>
Date: Tue, 8 Nov 2005 11:07:37 +0000 (GMT)
Hi VT, I strongly believe that replay tools are NOT an effective way to test an IPS: 1) Replay tools are an unfair way to compare vendors. There is no measure as to the speed at which an IPS vendor responded to the original vulnerability. These .pcap files have been around for months or even years, giving plenty of time for vendors to catch up and write signatures to stop them. It's all very well having an IPS that responds favorably to a replay tool, but if certain signatures took days, weeks or even months to write, then this is not a fair way to compare device A with device B. It can be difficult to get hold of, but try and get the 'time to respond' stats from your proposed IPS vendor. 2) Replay tools do not test variants. Although the pcap content may reflect a specific exploit, what about all the other exploits that abuse the same vulnerability? For example, the 50+ odd variants of Blaster, Slammer or Code Red? These test tools usually include pcaps for one particular variant only. A good freeware tool to test variations is Metasploit, or if you have spare cash, Canvas is a worthwhile investment. 3) Replay tools do not test ability of a device to withstand volume. When worm/virus outbreaks happen, you don't just get a single packet in .pcap fashion that comes in, trips a signature, and gets blocked. You usually get several million of them. This is where it is important to test any rate-based features of the device. Also make sure these rate-based features don't block valid traffic. Plenty of freeware tools available - Apache Benchmark, nmap, Nessus, hping2 and a SYN Flood tool called Juno. What's more, these tools do not rely on .pcaps, so are a lot more real world. Be warned that replaying an identical pcap several million times is not an accurate way to rate-test, seeming all L2/L3 information is identical in each packet (eg no change in SEQ). Devices that are good at DDOS tend to be just as good at withstanding sudden, large propagation of worms. 4) Replay tools do not test IPS avoidance. Well, you may get vendor-supplied pcaps like blaster_with_fragments.pcap, but there are so many other ways to evade an IPS, and can you be sure that the vendor supplied .pcap is a reflection of real world traffic? Get hold of fragroute, tcpsic, plus nessus has some good evasion options. 5) Replay tools do not test zero-day protection. This is the fun bit. How do you test a network IPS protects you against the shape of things to come? If you have a research team that can generate signatures to protect you quickly, then great. But there's still a window of opportunity during which your security can be breached. If an IPS is firing back events such as Sasser.W32/A blocked, or Code.Red.B.W32/Z, then chances are, it's not an anomaly based system that will give you much in the way of zero-day protection. But if it's firing events like 'CIFS Field Too Long' or 'HTTP Header Contains Illegal Characters', then this is indicative of the machine having good anomaly/zero-day procetion, rather than specific signatures for specific pre-historic viral events... ;) IPS testing is a big task, and needs big tools - to do things properly, take a look at the NSS testing suite, for example - http://www.nss.co.uk/utm/appendix_a/appendix_a.htm In conclusion (I hear sighs of relief...): * Replay tools cannot be solely relied on to test an effective IPS * Think about what you want your network IPS to do - content-based checks are important, but equally as important are access control and rate-based ability. * A network IPS will never provide 100% perimeter protection. Always invest in extra security layers, especially host-based, to ensure that anything the IPS lets through does not cause problems * If you buy an IPS on the merits of tcpreplay results, you risk being hit with a zero-day threat or DoS condition, and losing your job. * Treat any vendor that promotes testing with a replay tool with caution (I learnt the hard way..) Hope this helps, Regards, Matt --- Arun Vishwanathan <arun.vishwanathan () nevisnetworks com> wrote:
Hi VT, I have used IDSInformer myself for testing and it is a very good product. There is a similar free tool (but lacks certain features) called Tomahawk which was released by Tippingpoint some time back. (http://tomahawk.sourceforge.net/) The working of these tools is very simple. You have to assign two interfaces. The tools consider one interface as "client" and other interface as the "server". The PCAP can be easily split into two parts, client traffic and server traffic. Consider the following simple packet sequence (A and B are IP addresses). 1. A -> B SYN (client) 2. B -> A SYN-ACK (server) 3. A -> B ACK (client) Packet 1 is first sent out on client interface. The packet is expected to arrive on interface 2 within a certain timeout. On receipt of packet 1, packet 2 is sent out on interface 2. Then packet 3 is sent out on interface 1 on receipt of packet 2 and so on. They make the IDS believe that it is seeing a real traffic situation. In informer, you can change the MAC, IPs, Sport, Dport of the packets. In tomahawk you can only change the IPs at present but if you want to you can easily modify the code as its very simple. There is no need to configure any networks on the interfaces etc. Infact the IPs, MACs can be spoofed because it really doesn't matter. Tomahawk has one limitation that it cannot test a Layer 3 device because it lacks support for specifying the source gateway MAC and Destination gateway MAC. It can test only Layer 2 devices. Informer can be used in both L2 and L3 situations. In my opinion, both tools are great. I have used and am using both tools extensively. Informer also has an evaluation version. You can download it and try for yourself. For both the tools very little configuration is required. Hope I was able to clear some of your doubts. Regards, Arun -----Original Message----- From: vendortrebuchet () comcast net [mailto:vendortrebuchet () comcast net] Sent: Sunday, November 06, 2005 6:11 AM To: thaywood () karalon com; focus-ids () securityfocus com Cc: Tony Haywood; pen-test () securityfocus com Subject: RE: Intrusion Prevention requirements document This sounds like a very viable solution that will allow for testing. I assume that it replays both the stimulus and response of any conversation and does not "fingerprint" the packets at any layer with the host OS TCP/IP stack (e.g. change of window size, TTL, etc)? Does the product automatically adapt to replay source and destination traffic based upon reading a libpcap file or do you have to configure the networks per card? Has anyone else used this or a similar product in their testing or other security product tests? What issues did you encounter? Thanks for the feedback, -VTOne of the ways that you could test safely is byusing something likeTraffic IQ Pro or a similar product. It is astateful traffic replay tooland can be used to test any inline or packetmonitoring device.The product uses two network cards and so thelibrary of over 700 normal andthreat traffic files can be replayed statefullywithout the need to connectto a live target system. This allows for liveproduction systems to betesting for the correct configuration reallyquickly and easily.I have been involved in working in this area for anumber of years now andmy previous company was Blade Software where Ideveloped IDS Informer andFirewall Informer to provide similar testingcapabilities.Information on Traffic IQ Pro is available belowshould you want to take alook.
http://www.karalon.com/Karalon/TrafficIQ/TrafficIQ.htm
Working with testing labs and a number of securityand networking vendorshas enabled Traffic IQ Pro to be a really usefultool for anyone who wantsto check the configuration of their firewalls,IPS, IDS, routers, switchesetc and see how those devices perform underdifferent scenarios.Tony Tony Haywood www.karalon.com -----Original Message----- From: vendortrebuchet () comcast net[mailto:vendortrebuchet () comcast net]Sent: 29 October 2005 20:40 To: focus-ids () securityfocus com Subject: Re: Intrusion Prevention requirementsdocumentAnother question for everyone, When you brought in each vendor for evaluation,did you configure a testnetwork for them or did you use your productionnetwork? My 1st concern iskeeping my job :o) If I test in production, Icould impact productiontraffic. If I don't test in production, how can Ibest ensure that I won'thave problems with custom applictions, older IPstacks which could be anissue if RFC compliance checks are done, etc. The vendor answer is always, "don't turn onblocking and just monitor." Isthat a reality? I'd like some testimonials tothis and some real lifeinstances of what has been done from unbiasedsources.Thanks, VTAll, I work on a team that manages signature andbehavioral based intrusiondetection systems today. We have been taskedwith reviewing IPS (orwhatever vendor name acronym you prefer) in '06.Our normal process
=== message truncated === ___________________________________________________________ To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. http://uk.security.yahoo.com ------------------------------------------------------------------------ Test Your IDS Is your IDS deployed correctly? Find out quickly and easily by testing it with real-world attacks from CORE IMPACT. Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 to learn more. ------------------------------------------------------------------------
Current thread:
- Re: Intrusion Prevention requirements document vendortrebuchet (Nov 03)
- RE: Intrusion Prevention requirements document Tony Haywood (Nov 07)
- RE: Intrusion Prevention requirements document Andy Cuff (Nov 08)
- RE: Intrusion Prevention requirements document -Apology Talisker (Nov 09)
- <Possible follow-ups>
- RE: Intrusion Prevention requirements document Arun Vishwanathan (Nov 07)
- RE: Intrusion Prevention requirements document FinAckSyn (Nov 09)
- RE: Intrusion Prevention requirements document Tony Haywood (Nov 10)
- Re: Intrusion Prevention requirements document Mike Frantzen (Nov 14)
- Re: Intrusion Prevention requirements document Bob Walder (Nov 10)
- RE: Intrusion Prevention requirements document FinAckSyn (Nov 09)
- RE: Intrusion Prevention requirements document vendortrebuchet (Nov 07)
- RE: Intrusion Prevention requirements document Tony Haywood (Nov 10)
- RE: Intrusion Prevention requirements document Chris Ralph (Nov 14)
- Re: Intrusion Prevention requirements document ADT (Nov 16)