Re: IPS in the Enterprise UTM Firewall testing results

[Matt Jonkman <jonkman () jonkmans com>]

I'd just summarize it all up by saying that IPS is ready for prime time,
but not ready for auto-pilot configuration. It still requires
site-specific config and testing, and an experienced human running it.

Hitting the "Turn on IPS" button is just not feasible at this point in
time. In a few years, maybe. But certainly not now. (With any engine)

Matt

Joel M Snyder wrote:
> I wouldn't necessarily say that catch rates are disappointing.  With
> IPS, it is very difficult to say what a good catch rate is.  Clearly,
> the ISS box "caught" more things than all of the other guys, but
> remember that the purpose of an IPS is to handle that narrow window
> between problem and patch--if you are relying on your IPS to block SQL
> Slammer, you've got some major architectural conceptual errors in your
> network that IPS won't help you with.
>
> I was pretty careful NOT to make any pejorative statement about the
> catch rate (except to say that relative catch rates give you relatively
> 'better' IPS), and I think that we ALL have to be careful in that area.
>
> I don't believe that anyone can credibly put a stake in the ground and
> say "an IPS must block these specific attacks" and then defend that
> position.  This is very different from, say, A/V or firewall, where
> there's a much clearer black-and-white line about what you need to support.
>
> Clearly there are some pathological environments where an IPS somehow
> substitutes for a firewall and where 6000 signatures is the "right
> number" to have.  But in enterprise deployments, it's very unclear to me
> how to adequately test an IPS for coverage.  I can do performance easily
> enough, but checking coverage (which is what the Mu-4000 does) just
> seems quite dangerous.
>
> Anyway, I think that it is useful to see the comparative values on IPS
> catch rate, but I would not go so far as to say that having an average
> catch rate in the 30% to 40% range is "bad" or "good" for these products.
>
> I want to distance any testing we do from the bogus premise that you see
> in tests like the ICSA certifications where they pick specific attacks
> and say that you must block these.  To me, that's not supportable.  It
> may be in an IDS, but IDS and IPS are entirely different beasts, and we
> were testing these products as IPSes, not IDSes.
>
> jms
>
>
>
> Ravi Chunduru wrote:
> > this is really a great report and i am sure lot of effort has gone
> > into this. catch rates and perforamance is really caught my eye.
> >
> > Catch rates are really disappointing across the board except for ISS.
> > i do understand that client attack detection is new, but even the
> > server side catch rates are awfully low. i understand that these are
> > expensive boxes. i did not see any vendor responses  on low catch rate
> > and performace.
> >
> > is this due to technology limitation or is it that devices tested are
> > not up to mark?
> >
> > Ravi
> >
> > On 14 Nov 2007 15:28:18 -0000, jms () opus1 com <jms () opus1 com> wrote:
> > > After months and months and months in the lab, a huge UTM test I did
> > > for Network World is now available (for free, folks, for free) on
> > > their web site.  I apologize in advance if you have to click 800
> > > times to read the whole 19,000 words, but here goes:
> > >
> > >
> > > Main story starting point:
> > >
> > > http://www.networkworld.com/reviews/2007/111207-utm-firewall-test.html
> > >
> > >
> > > Just the discussion of IPS in the UTM firewall/enterprise space:
> > >
> > > http://www.networkworld.com/reviews/2007/111207-utm-firewall-test-ips.html
> > >
> > >
> > >
> > > Chart on catch rates based on Mu-4000 testing:
> > >
> > > http://www.networkworld.com/reviews/2007/111207ips.html
> > >
> > >
> > > If you're not sure that enterprise should even be running IPS in
> > > their firewalls, you can click on the link below for a header page
> > > which has further links with some discussion on the pros and cons of
> > > that issue:
> > >
> > > http://www.networkworld.com/buyersguides/guide.php?cat=865480
> > >
> > >
> > > Enjoy or not, as you see fit.
> > >
> > >
> > > jms
> > >
> > >
> > > -- 
> > >
> > > Joel M Snyder, 1404 East Lind Road, Tucson, AZ, 85719
> > >
> > > Senior Partner, Opus One       Phone: +1 520 324 0494
> > >
> > > jms () Opus1 COM                http://www.opus1.com/jms
> > >
> > >
> > > ------------------------------------------------------------------------
> > > Test Your IDS
> > >
> > > Is your IDS deployed correctly?
> > > Find out quickly and easily by testing it
> > > with real-world attacks from CORE IMPACT.
> > > Go to
> > > http://www.coresecurity.com/index.php5?module=Form&action=impact&campaign=intro_sfw
> > >
> > > to learn more.
> > > ------------------------------------------------------------------------

-- 
--------------------------------------------
Matthew Jonkman
Emerging Threats
US Phone 765-429-0398
US Fax 312-264-0205
AUS Phone 61-42-4157-491
AUS Fax 61-29-4750-026
http://www.emergingthreats.net
--------------------------------------------

PGP: http://www.jonkmans.com/mattjonkman.asc



------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it 
with real-world attacks from CORE IMPACT.
Go to http://www.coresecurity.com/index.php5?module=Form&action=impact&campaign=intro_sfw 
to learn more.
------------------------------------------------------------------------