RE: IDS Testing tool

["BLADE Software - Chris Ralph" <chris.ralph () blade-software com>]

Group,

In response to this latest thread I would like to put a solution providers
perspective on things:

<sales pitch>
Testing an in-line IDS is extremely simple if you use IDS Informer, its easy
to install and use, totally safe when used correctly, has built in detailed
reporting that can identify not only that an inline IDS detected an attack
but at what packet it did so and can also be used to validate the
architecture and management of the entire IDS.
</sales pitch>

Granted there is a cost associated with IDS Informer but like most things in
life, everything is negotiable and when compared to the alternative of
sourcing and running the attacks and provides a very high ROI.

Currently the database is limited to around 650 attacks with new attack
packs coming soon which as has already been pointed out, is over 4 times the
amount offered by other solutions and can easily be augmented by the user
through the Informer Development Kit which will convert virtually any packet
capture for use in the product.

When comparing like technologies especially when validating an IDS, you need
to consider a few points which I don't think have been raised here.

Possibly the most important aspect is the security concerns associated with
running actual exploits on a production network.  In most organisations this
would simply be unacceptable and so limits a number of tools to
pre-production use only.

Then there are the configuration issues.  Tools such as TCP Replay can be
difficult to install and use for a number of users who are unfamiliar with
its requirements, the individual nuances of their current Unix configuration
or TCP/IP (not everyone testing an IDS is as technical as we would expect).
Compare that to a tool such as IDS Informer which uses a simple installation
routine on a platform that just about anyone not only has access to but is
comfortable using, is ready to run in minutes and is endorsed by the
majority of the major IDS vendors and the choices become more limited.

Overall setup time is another issue.  For instance how long would it take to
source and run 280 backdoors or 190 HTTP attacks that complete? That means
not only getting working versions of the attacks/backdoors but running on
the correct and in some cases, vulnerable platforms. With IDS Informer its
all there in individual attacks so there is virtually no setup time.

Then there is the need (sometimes) for a target host in a potentially secure
environment, not always easy to find.  IDS Informer is both the source and
the destination negating the need for any additional hardware. 

And what about the question of repeatability.  How easy would it be to run
an identical test 5 mins, 5 days or even 5 weeks later?

Lastly there is the attack library itself which enables extremely
granular testing using complete attacks (3-way handshake, pre and post
attack activity and reset/four-way teardown where relevant) unlike the brute
force method of sending large volumes of pre-captured traffic from sources
such as those from Defcon or the Honeynet challenge which contain multiple
exploits and normal traffic combined making it very difficult to determine
which attack your IDS is detecting, or not as the case may be, to maintain
state or to run specific tests.

For instance: how easy would it be to run BO2K between two virtual hosts
through an inline IDS using multiple destination ports?  In IDS Informer
this would take seconds.  Isolating the packets in publicly available packet
captures or running the actual code will take a great deal longer and may
not be possible at all and this simple test can be used to prove that a
number of IDS solutions have difficulty detecting this and other backdoors
when they are using well know ports or when they are transmitted with
unusually fast or slow (but not impossible) packet rates?

Due to IDS Informers unique ability to not only replay pre-captured network
sessions through multiple cards but to do so without the need for an IP
stack to be bound to any card with guaranteed packet delivery, it can be
placed in secure environments negating the need for any target device.

If you are looking for a complete solution then try combining your scanner,
exploit (pre-captured and converted using the Informer Development Kit of
course) or Informer Product Suite application with the Evasion gateway
giving you all the additional capabilities typically found in tools such as
Nikto and Fragrouter.

Ok so it could be argued that my opinion is a tad biased given that I have
been instrumental in the development of these applications and libraries but
having spent a number of years prior to this working in the security market
focusing on IDS and being frustrated by the difficulties in accurately
validating IDS (try teaching an IDS course and expect the students to take
for granted whatever you say without proving it, not fun!) I still think
that the Informer Product suite offers possibly the best overall solution
both technically and financially.

Chris
BLADE Software


-----Original Message-----
From: typhon --- [mailto:securecatalyst () hotmail com]
Sent: 15 June 2004 23:02
To: rgula () tenablesecurity com; focus-ids () securityfocus com
Subject: Re: IDS Testing tool


So, all in all, testing the IDP/IPS products, especially in in-line mode is
quite a challenge. There are some tools available out there like Blade's
IDSInformer (which only has around 600 signatures and most of them are DoS,
dDoS, Backdoor Detection rules), Core IMpact (Only has around 120 actual
exploits), CANVAS (even less exploits)...

It seems that a good IPS testing product with full blend of features and
complete attack list is what market needs... Attention entrepreneurial
minds!

I have not heard about the Fluxay, what about it?

Thanks, M/




> From: Ron Gula <rgula () tenablesecurity com>
> To: focus-ids () securityfocus com
> Subject: Re: IDS Testing tool
> Date: Sun, 13 Jun 2004 19:53:11 -0400
>
> At 10:58 AM 6/12/2004 -0700, ADT wrote:
> > On Fri, 11 Jun 2004 01:13:29 -0400 (EDT), Anton A. Chuvakin
> > <anton () chuvakin org> wrote:
> > > > Is anyone aware of any open source equivalent of Blade's IDS Informer
> > > > tool to test IDSes? I am aware that TCPReplay can be used to test
> > IDSes
> > > > but then we will need to make actual attacks at least once to capture
> > > > the traffic. Any help would be appreciated.
> > >
> > > What's wrong with just blasting it with a vuln scanner? Nessus will
> > > generate a lot of noise in most NIDSs and can even be tweaked for more
> > > "noisyness"
> >
> > Well think about it... a good IDS which limits the number of false
> > positives should detect the actual exploit.  A vulnerability scanner
> > is supposed to check for the vulnerability, *not* to run the actual
> > exploit, b/c then it may crash/root/etc your own box.  Hence, an
> > exploit should look different then a vulnerability check.  Therefore,
> > using Nessus or other vulnerability scanners are a crappy way of
> > testing an IDS.  (Of course if you've got a crappy IDS, then perhaps a
> > crappy test methodology is ok.)
> >
> > With that in mind, you can either use Blade's IDS Informer or roll
> > your own solution using tcpreplay.
>
> I'd say using vuln scanners is far from crappy, but surely not complete.
> It depends what you are looking for really. Vulnerability scanners do a
> wide variety of things from port scanning, host enumeration, TCP/IP
> fingerprinting, service probes (like SNMP, RPC, Netbios, .etc) and so on.
> Of course, none of those may constitute an actual intrusion, but 80-90%
> of most NIDS tend to focus on those activities.
>
> If you really want to see exploits in action, I would recommend using
> CORE, Metasploit, Fluxay, or some other tool that allows you to coax
> root or admin from an active exploit. Finding a bunch of remote root
> exploits on packetstorm, installing the vulnerable versions of those
> daemons on your target system and then launching the exploit in front
> of your NIDS is something everyone should do once or twice. You'll be
> very surpirsed what your NIDS see and don't see.
>
> I'd also recommend you try to bind a shell to some obscure high port with
> netcat and see how your NIDS reacts. Lots of UNIX and W2K attacks invoke
> a listener on some other port. If you really want to make it difficult,
> put the listener on port 80.
>
> And lastly, using TCPreplay with the traces from Defcon CTF or the
> honeynet challenge can also present your NIDS with a source of traffic.
>
> Ron Gula, CTO
> Tenable Network Security
> http://www.tenablesecurity.com
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------------
>
> ---------------------------------------------------------------------------

_________________________________________________________________
MSN Toolbar provides one-click access to Hotmail from any Web page - FREE
download! http://toolbar.msn.click-url.com/go/onm00200413ave/direct/01/


---------------------------------------------------------------------------

---------------------------------------------------------------------------








---------------------------------------------------------------------------

---------------------------------------------------------------------------