RE: Intrushield vs. ISS once more...

["Murtland, Jerry" <MurtlandJ () Grangeinsurance com>]

David,

No offense, but it seems that you feel that the ISS ProventiaG product is
perfect, or at least you describe it as such.  In your opinion as a resident
SME, what do you see as shortcomings of the ISS Proventia product?  I have
some experience with the ISS Proventia G as I recently reviewed it for my
company over a period of 4 months and saw a few issues with it to include
false positives, packet capturing limitations (does not do deep packet
inspection, only captures first X amount of bytes), and I believe that the
reporting that has come so far in becoming more customizable is about to go
back to static reporting.


I'm not against your product. In fact, it was one of the final 2 products
(out of 15) that we decided to go with, but didn't win.  Pricing was a bit
steep, but I can acknowledge that with all of the company's research,
salaries to be paid for that research doesn't come cheap.  Marketshare
basically reflects the longevity of the product vs. competitors, so I'm
can't buy popular opinion.  I know it is a decent product, but it's not
perfect.  I'd just like to hear your opinion on where improvements could be
made.  Ultimately, I think pricing, complexity/intimate product knowledge
requirements and a few false positives with bad reporting are the largest
hurdles that the product faces.


Jerry


-----Original Message-----
From: Maynor, David (ISS Atlanta) [mailto:dmaynor () iss net]
Sent: Friday, December 31, 2004 4:16 AM
To: Eric Hines; Brito, Nelson (ISS Brazil); Murtland, Jerry; Jacob
Winston; focus-ids () securityfocus com
Subject: RE: Intrushield vs. ISS once more...


Please forgive the length of this post. The note I am replying to has
several misconceptions that require both a few words and a little bit of
time to clear up. It also contains a lot of specific vendor cheerleading
but I still think it is an interesting read.

> 1) ISS engineers have told me time and time again that turning on the
> "packet capture" capabilities, which I think ANY IDS/IPS should
> automatically be able to provide without turning on, was not
recommended.
> That is, the performance hit would be too great when these features
were
> enabled. I don't understand, does ISS really expect us to believe ISS
when
> it says it's detected a particular attack? By default, not allowing the
user
> to see the packet is a pretty bold statement from ISS that it doesn't
expect
> to see any false positives. When ISS acquired NetworkICE I had really
> expected Robert Graham to come in and clean up RealSecure. 

I will address the ISS confidence issues first. A lot of research,
engineering and QA time goes into every check shipped. If it was a
recent sales pitch you should have gotten the speech about the X-Force
being a true differentiator because of the amount of research done for
every check. Far more research goes into everything ISS ships than any
competitor I am aware of. Keep in mind I have only been with ISS since
May and before that I was a technical engineer in network security for
an extremely large university and my emphasis was IDS/IPS. Because of
this I have seen almost ever security sales pitch on the planet. Almost
every vendor that would pitch would spend a large amount of time on
features like speed and ease of use but would only touch on their
ability to develop security content, if at all. ISS views the ability to
create top notch content to be as important as the speed and the ease of
use of a product. What good is a fast, easy to use product (not saying
that ISS isn't fast or easy to use) that lets the next slammer or sasser
into your network? So yes, due to the protocol parsing and the amount of
effort that goes into every check, we do not expect to see false
positives. That is not saying it does not happen every once in a blue
moon, but we are surprised when it does and quickly fix it. If you are
skeptical then you are not familiar with the way ISS develops content.
It impressed me so much I joined the company. 

Back to the topic of why ISS should be trusted...

Since the checks are geared toward protection of the vulnerability
rather than detection of a specific exploit we have found what many
people think is a false positives are actually new worm variants or
custom written versions of an attack against vulnerabilities. As far as
"believing" ISS when an attack is detected, it is expected that a
customer has confidence in a box they purchase and have done due
diligence in the testing phase. Several people, including myself, have
commented on this list about how to actually test the security content
from a vendor. I think this type of testing should be mandatory before a
purchase and will quickly expose bad security content coverage. This
type of testing will also breed confidence in the belief that when ISS
ships protection, protection is what the customer gets.   

On the subject of the information passed to you by SE's?

Please provide me the names of the sales engineers, off list if you
prefer, and the date you met with them. I have a sneaking suspicion the
pitch you saw is old or happened some time ago since you mentioned
RealSecure and not Proventia and a lot of the problems you bring up were
addressed by Rob Graham (or Big RobG as we like to call him).
 
There has been 3 distinct phases that packet caps in ISS products have
went through. 

The oldest phase captured the data and wrote to the sensors disk. The
file was stored in a "Sniffer" format or could be configured to use
tcpdump format. This could eat up a lot of space and could become a
performance/maintenance nightmare. 

The current method in use improves the original method by allowing the
captures to be sent to the management console. This is configured on a
per-event basis. One of the reasons it is done on a per-event basis is
it allows commonly seen attacks to be filtered out if you feel confident
the ISS protection is stopping the threat. Who really wants 200 gigs of
slammer traffic lying around? When a configured event is triggered it
will send the event information and the traffic to the management
console where the admin has a choice of what to do with it; save it or
discard it. Since ISS truly writes checks for the vulnerability and not
specific exploits or worms this feature has allowed ISS to quickly
identify new variants of attacks. There is a story about an ISS customer
complaining that a check was false positive triggering and should be
disabled. This packet-capture feature allowed     X-Force engineers to
review to traffic and determine it was a new worm variant that was using
the same vulnerability but a different propagation method that other
vendors didn't have detection for yet. It would be hard to convince
customers to provide this type of feedback if the performance was taking
major hits.

The newest method which should be released soon has adopted a modified
libpcap interface. This allows an admin to ssh to a sensor and grab the
packets with something like tcpdump or tethereal.

As far as a performance hit there is a bit but a lot of it depends on
the average packet size of the network and how often an attack is being
logged.

The statement "the performance hit would be too great if these features
were enabled" sounds like there is some information being left out? For
instance is your network running at a full gig with an average packet
size of fewer than 100 bytes and you want to record every event in and
out on certain type of box that is not really rated for those speeds?
This shouldn't be the case and Proventia should be tuned to the
customer's network to start weeding out commonly seen events that have a
high confidence of being blocked. But for the sake of argument let's
just say you dropped a Proventia G1000 on this network with no tuning
and everything turned on. I could see this causing a performance hit but
in this case storage and analysis of the raw events by the customer will
be a far larger concern than the speed in which they are collected.

Aside: For an interesting take on something like this read up on a
Lancope product called Therminator. It incorporates a process they call
"psychic packet prediction." This attempts to maintain a certain buffer
of every connection. I can't remember the number; I think 50 packets or
so. When something in that buffer causes an alarm the entire buffer is
saved so not only do you get the packets that caused the alarm, you get
a certain amount ahead and behind it. This helps with forensics greatly.
Pretty cool stuff.

> 2) The Trons functionality you speak of, again, is not recommended for
use
> due to the impact that has on performance when using too many Snort
> signatures.


The main component to ISS network protection is a protocol analysis
engine we call PAM (Protocol Analysis Module) and should not to be
confused with Pluggable Authentication Module. PAM is what actually does
the protocol breakdown and analysis for what we deem "bad traffic." The
initial version of TRONS sat outside of PAM. Per Rob Graham, the
performance sucked. It was designed to use with less than 50 rules and
because it was outside of the main PAM module rule processing was
severely impacted.

Luckily that version is dead and gone. 

The latest version of TRONS is integrated into PAM and has done away
with the originals performance problems. Unlike the original 50 rule
design limit of the original, the new TRONS is designed to let customers
import hundreds of rules if not the entire snort ruleset. In case you
were thinking of mentioning that customers would more than likely have
to port all their old rules to a new format; let me set your mind at
ease. The actual format has changed very little. Where as before a user
would do something like:

        trons.rule=alert tcp any any -> any ...

The new version is a simple change like:

        pam.trons.rule=alert tcp any any -> any ...

Pretty easy, IMHO. 

> I reiterate that these are not my opinions, rather, what was said to me
when
> I last sat in on an ISS sales presentation with several SE's present.

As I said before please send me the names of the SE's you met with and
the date and I will see this misunderstanding gets cleared up.  

--
David Maynor
Research Engineer, X-Force R&D



--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from 
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
--------------------------------------------------------------------------