RE: Testing IDS/IPS Signatures

["BLADE Software - Chris Ralph" <chris.ralph () blade-software com>]

There are other tools such as IDS Informer and the Evasion gateway from
BLADE Software that go way beyond the capabilities of the applications
already listed and are much easier to use.

I agree with the ACT_GATHER_INFO comments in relation to scanners.  Scanners
rarely run a complete or even real attack and as such are extremely limited
in signature validation.  Take something very simple like a backdoor, can a
scanner be used to detect it? Yes, can a scanner be used to validate the IDS
signature in a both successful and unsuccessful state, no.

Scanners have a job, they determine vulnerabilities in active hosts they
were never designed to validate an IDS and have limited use in this respect.

The evasion gateway was designed to provide all of the functionality of
fragrouter on a Windows platform combined with the obfuscation techniques of
Nikto but on a network level for all HTTP based traffic.

IDS Informers attack are fully stateful (they contain three way handshakes,
pre and post attack activity and session resets/four way tear-downs) and can
be used in conjunction with the bi-directional transmission option to
validate all IDS solutions including inline.
  
Chris
BLADE Software

-----Original Message-----
From: ravivsn () www rocsys com [mailto:ravivsn () www rocsys com] 
Sent: 29 May 2004 7:32 AM
To: rgula () tenablesecurity com
Cc: focus-ids () securityfocus com
Subject: Re: Testing IDS/IPS Signatures

True, Nessus can help in testing signatures but IMHO, it has limitations.
All the nasl scripts in Nessus do not really attempt to run exploits, most
of them are ACT_GATHER_INFO means they look only if particular port is
opened or checks for an version in the banner received.
Also to test all the signatures you need systems which has those
vulnerabilties. If not, Nessus is going to fail to show up the results.

I have bit experience in testing IDS/IPS signatures. I used Nikto,
libwhisker and mutate2. Mutate2 is a good tool which really tests anti
NIDS tactics.

As far as snot/stick are concerned, they are not intended to test
signatures. These tools triggers lot of false positives by generating
packets matching the patterns of snort signatures. In a way these tools do
help to tune singatures into good shape such that they wont add fire to
false positives.

 Snot/stick will effect IDS like snort but they fail to influence IPS
because they lack threee way hand shake and IPS which might have stateful
inspection will easily block snot generated packets.

I did some work over this and developed e-snot, which when run on snort
gave lots of false positives, I can say for almost all signatures there is
a false positive.

Best Regards,
-Ravi
ROCSYS Technologies Ltd.,
http://rocsys.com
mail me to : ravivsn () rocsys com

> Anyone testing an IPS should attempt to use the denial of
> service features in Nessus and NeWT to see what is in fact
> being prevented. Nessus and NeWT contain a wide variety of
> DOS checks which perform fairly invasive tests.
>
> Nessus and NeWT also have a variety of anti-NIDS evasion
> features built in. For example, you can perform a variety of
> web vulnerability scans, and have them use URL encoding,
> TCP desynchronized packets and fragmentation. Although using
> a vulnerability scanner to test a NIDS is an imperfect test,
> comparing what a NIDS picks up when evasion is and isn't used
> during a scan is extremely enlightening.
>
> Most people know that Nessus can be obtained from
> www.nessus.org but they may not know that NeWT is also available
> as a complimentary download from www.tenablesecurity.com.
> NeWT is available for Windows XP/2000 and can scan any machine
> on the local "Class C" network. It performs the same security
> checks as Nessus, but has it's own interface, reporting and
> usability features. NeWT Pro is the commercial variant which
> has no local "Class C" scan limitation. If you have an IDS or
> IPS in a lab or on a small DMZ, you can use NeWT to launch
> your tests from any available Windows laptop or server.
>
> Ron Gula, CTO
> Tenable Network Security
> http://www.tenablesecurity.com
>
>
>
> At 06:30 PM 5/27/2004 -0800, Securecatalyst wrote:
> > Hi All,
> >
> > I want to learn if anyone knows any particular tool or product to test
> > and validate IDS/IPS rules and signatures?
> >
> > I know Snot / Stick / Mucus-1 can do a good job however they can not
> > test the signatures when the IDS/IPS does a stateful-inspection. They
> > simpy import the SNORT signatures into packet and inject into the NW to
> > test the rules. However, they do not establish TCP 3-way handshake and
> > stateful engines (specifically for TCP, not UDP/ICMP) simply ignore
> > them.
> >
> > I think Blade Software have some good marketing documents but I also
> > heard that their signature set is not complete to test all. Anybody any
> > experience with this?
> >
> > Further, is there any other way to validate the IDS/IPS signature other
> > than running the attack itself against a vulnerable machine? I think
> > vulnerability assesment tools does not help, due to similar reasons
> > with Snot/Stick.
> >
> > I particularly wonder how TippingPoint, Intruvert, Toplayer and
> > OnseSecure verifies their signatures? Or, do they really verify? If
> > they did, they wouldn't be this many false-positives, right? I know
> > some vendors simply take SNORT signatures and put it into their SNORT
> > modified engine but I am getting lots of complaints around SNORT's
> > noise and false positives.
> >
> > Your input will be highly appreciated.
> >
> > Cheers,
> >
> > --------------------------------------------------------------------------
-
> > --------------------------------------------------------------------------
-
>
---------------------------------------------------------------------------
>
---------------------------------------------------------------------------




---------------------------------------------------------------------------

---------------------------------------------------------------------------




---------------------------------------------------------------------------

---------------------------------------------------------------------------