RE: Help in evaluating Inline IDS/IPS solution

["Brian Laing" <brian.laing () blade-software com>]

Ravi,
        Having come from an IDS vendor and now offering products around IDS
and firewall auditing, I would be happy to answer some of your questions as
well as point you in our direction.


Do IDS vendors really test the signature against the vulnerable 
applications, hardware platform of the application and version of
application before 
releasing the signature? 
[Brian] This really depends on the vendor and the signature.  Some
signaturers are written without an exploit existing.  For those that have an
exploit seom vendors do this some do not, some are using our products since
our product can put this sort of traffic on the wire.

Do the IDS vendors claim this?
[brian]  I have not seen andy vendor claims on this doesn't mean its not
there just I have not seen it

If so, what is it I need to look for?
[brian]What I would look for is frequency of updates.  Also if you can
extend your evaluation to vover several updates of the product you will be
better off.  I have seen many times in the field IDS doesn't detect attack,
make update it does detect, apply update it goes back to not detecting
attack.  I have seen signatures change severity, or drop off all together.
I have seen the packet reassebly work in one version and be broken in
another.  Only testing across multeiple releases can you see this.  That is
why we recommend testing EACH update.


 From sensor technology perspective, I find that all the vendors 
seems to be having similar capabilities. But, I am trying to see the
continued 
support on new attacks and vulnerabilities found.

[Brian]  Yes I would agree many of the vendors SENSORS are very similar.  I
think customers now need to focus a lot more on management of the IDS.  This
was less true several years ago when the Sensor was the main thing, but now
most sensors are VERY close to each other in performance, detection, and
other features.  Managing those features etc. is now the biggest
differenatiator I am seeing customers ask for.  Followed by speed!
      One vendor claims that they have 5 dedicated analysts looking at 
the vulnerabilities   and updating signatures (if needed). Another vendors
claims that 
they have more than 20 analysts doing this job. Can this be considered in my

eval?
[brian] I would not consider the number of analysts doing the work but the
frequency of updates and the quality of updates.  If one vendor has 10000
people working on the problem but updates are sporadic, difficult to
implement etc. then those 10000 people are useless.  If however one vendor
has 5 people and is regular as clock work on updates that is the route to go


 Is it that other vendor exaggerating the number of resources they have for
this job.
[brian] creative counting has always been part of this market, just look
back at the way signatures were counted both for early IDS and vulnerability
assessment.  One vendor counts a single teardrop (but checks for 15
iterations) while another vendor counts each iteration as a different
signature.

  Performance:
      What is the best metric to look for? I feel HTTP1.0/1.1, SMTP, 
IMAP, NNTP, TELNET, POP3 connection rate and UDP throughput for different 
sizes is good metric. Is there anything should I look for?
[Brian]  I see this as really being several thigns that need to be tested
for
1. speed how much raw bandwidth can the sensor handle without dropping
stuff.  This is especially improtant in an inline IDS as dropped packets
don't make it regardless of attack detection.  Knowing the protocols on your
segment can help immensly in running your test as a 100% http traffic
segment is a lot different then a network with a variety of protocols to
assemble and analyze.
2. attack detection what do I detect at close to 0% network utilization once
you know this then you can step up to 
3. Attack detection under load  Various network loads to see when it looses
attacks vs just dropping packets.  Dropping packts and missing attacks are
two different beasts all together.
4. management, it can be the best sensor in the world but if you can not
manage the number of sensors you have and the number alerts you receive then
the sensor is useless.

      Are there any labs, which provide testing facilities for testing 
IDS/IPS with latest vulnerabilities and with real vulnerable applications? I
am really 
looking for lab which provides facilities and allows us to test the IDS/IPS 
solution on regular basis.
[Brian]  I am not aware of labs that let you walk in and test what ever you
want with this sort of test.  We do have that sort of facility but we only
have it open to a few people and its not available to the general public.
However you can use our software to simulate 100% accurate attacks between
two points using our IDS and Firewall informer products.  If you have any
questions about them please don't hesitate to drop me an email

Cheers,
Brian

-------------------------------------------------------------------
Brian Laing
CTO
Blade Software
Cellphone: +1 650.280.2389
Telephone: +1 650.367.9376
eFax: +1 650.249.3443
Blade Software - Because Real Attacks Hurt 
http://www.Blade-Software.com
-------------------------------------------------------------------


-----Original Message-----
From: Ravi [mailto:ravivsn () roc co in] 
Sent: Wednesday, June 04, 2003 9:41 PM
To: focus-ids () securityfocus com
Subject: Help in evaluating Inline IDS/IPS solution


Hi,
      My company plans to resell the Network Inline IDS/IPS solution to 
our customers and support
      customer. I was given task of evaluation of different solutions in 
the market. There are some
      questions asked by our customers and I would like to keep these in 
mind while
      evaluating the IDS solutions.

      Do IDS vendors really test the signature against the vulnerable 
applications, hardware
      platform of the application and version of application before 
releasing the
      signature? Do the IDS vendors claim this?  If so, what is it I 
need to look for?

      From sensor technology perspective, I find that all the vendors 
seems to be having
      similar capabilities. But, I am trying to see the continued 
support on new attacks
      and vulnerabilities found.
      One vendor claims that they have 5 dedicated analysts looking at 
the vulnerabilities
      and updating signatures (if needed). Another vendors claims that 
they have more
      than 20 analysts doing this job. Can this be considered in my 
eval? Is it that other
      vendor exaggerating the number of resources they have for this job.

      Performance:
      What is the best metric to look for? I feel HTTP1.0/1.1, SMTP, 
IMAP, NNTP,
      TELNET, POP3 connection rate and UDP throughput for different 
sizes is good
       metric. Is there anything should I look for?

      Are there any labs, which provide testing facilities for testing 
IDS/IPS with latest
      vulnerabilities and with real vulnerable applications? I am really 
looking for lab
      which provides facilities and allows us to test the IDS/IPS 
solution on regular basis.

      Thanks
       Ravi






-- 


The views presented in this mail are completely mine. The company is not
responsible for whatsoever.
------------------------------------------------------------------------
Ravi Kumar CH
Rendezvous On Chip (i) Pvt Ltd
Hyderabad, India
Ph: +91-40-2335 1214 / 1175 / 1184

ROC home page <http://www.roc.co.in>




----------------------------------------------------------------------------
---
INTRUSION PREVENTION: READY FOR PRIME TIME?

IntruShield now offers unprecedented Intrusion IntelligenceTM capabilities 
- including intrusion identification, relevancy, direction, impact and
analysis 
- enabling a path to prevention.

Download the latest white paper "Intrusion Prevention: Myths, Challenges,
and Requirements" at: 
http://www.securityfocus.com/IntruVert-focus-ids2
----------------------------------------------------------------------------
---


-------------------------------------------------------------------------------
INTRUSION PREVENTION: READY FOR PRIME TIME?

IntruShield now offers unprecedented Intrusion IntelligenceTM capabilities
- including intrusion identification, relevancy, direction, impact and analysis
- enabling a path to prevention.

Download the latest white paper "Intrusion Prevention: Myths, Challenges, and Requirements" at:
http://www.securityfocus.com/IntruVert-focus-ids2
-------------------------------------------------------------------------------