Recent Gartner IDS/IPS report

[Gary Golomb <gee_two () yahoo com>]

I wrote this late last week, but it bounced because of
user error (me). Jason asked if I wanted to repost it,
and since it still seems relevant (and it spurred some
interesting discussions on the ISN mail list), here
goes shot #2. It was also taken from ISN and posted to
Network World's forum, which elicited a response from
the author of the original paper. I'll forward his
response with comments inline to this list. I think
you all will like it...  

In the end though, one of the replies I got said it
best with the statement, "Arguing with Gartner is like
arguing with the National Enquirer." I think I now
understand just want he meant. 


==== Original Post:

Ok, this is going to be long. Also, this email is
being written entirely on my own impetus and
**definitely does not** reflect the views of my
employer. (In fact, I'll be surprised if I make it
through this without any bruises.)

Gartner, Inc. has recently released a document
authored by Richard Stiennon entitled, "Intrusion
Detection Is Dead - Long Live Intrusion Prevention."
(So I'm guessing we don't need to cover what that
document is about.) Gartner is self-described as, "For
20 years, Gartner's Research & Advisory services have
been recognized as the definitive source for objective
technology thought leadership." Ok, fair enough. I'm a
fair person and everyone makes mistakes.

Unfortunately, this is not Gartner's first mistake
along these lines. Here's a quote from paper now a
year and a half old (also from Gartner): 

"Intrusion Prevention Will Replace Intrusion
Detection. Enterprises should delay new large
investments in intrusion detection systems -- which
have failed to provide additional security -- until
intrusion prevention systems emerge that provide a
stronger defense against 'cyberattacks.'" 

No, this is not the first time Gartner has displayed
such a grotesque misunderstanding behind detecting and
defending against *real* threats, but this is
definitely the most horrible. 

So, for all those who take statements like the above
seriously, let's define WHY people use Intrusion
Detection technologies in the first place. 

Intrusion Detections systems are used for one reason.
It’s your last chance to be notified about a potential
break-in; a virtual safety net. Once an organization
has invested massive amounts of time, money, and
resources into setting up "PROTECTIVE" technologies
such as (but not limited to) firewalls, encryption,
authentication, proxies, gateways, PKI, VPN, access
control, virus detection/removal, etc... The IDS
serves the single purpose of sitting back and watching
over everything to see if people are still getting
though. And here's a curveball for you: After all the
protective technologies just described, attackers
(both automatic like worms/viruses and live people)
were/are STILL getting through! Whether it's because
of vulnerabilities in network designs, application
vulnerabilities, or unknowingly misconfigured devices,
they do get through. And this is why IDS's were
invented...

The main difference between an IDS and other security
devices is the fact that it's out-of-band, or passive
in nature. It passively watches all traffic looking
for SIGNS of attacks, compromise, or other misuse. The
key benefit to being out-of-band is that you have the
ability to flag traffic that looks even the slightest
bit "suspicious." If you have an IDS that is telling
you that too much is "suspicious," then tune it!
What's suspicious in one environment might not be in
another. Vendors try to compensate as best as
possible, but only YOU know YOUR environment the best!
Once it is flagged, it is usually logged and followed
up by automated processing, or people-based responses.

So, now that we're on relatively the same page when it
comes to ID, let's look at Gartner's reasons for
stating that we don't need this technology anymore. 

---
Statement #1 
"Contrary to the philosophy that it is impossible to
protect a network from all of the attacks leveled
against it..."
---

Ok, this one is more comical than anything else. It's
the first sentence in the document. By starting off by
telling us that it *IS* indeed possible to protect a
network from ALL attacks leveled against it, I had to
chuckle. It also set the stage for the rest of the
document. 

---
Statement #2
"The 'demilitarized zone' (DMZ) architecture has been
punctured by many exceptions to security policies. It
poses a threat to mission-critical services."
---

Since DMZ's [apparently] pose a threat to critical
services, Richard proposes (what he dubs as) a new
nomenclature and architecture for replacing the DMZ.
The new name is: The Transition Zone. (TTZ?) The way
TTZ works is by taking your public resources (like a
firewall, mail serer, or whatnot) and placing it on a
network that is logically between the Internet and
your internal network. This middle ground is separated
from the Internet via a firewall or gateway that
allows limited access to the public resources. There
is a second firewall that separates the TTZ from the
internal network which I presume is more restrictive. 

Interestingly enough, that's what the rest of the
world calls a "DMZ." I saw no difference between the
proposed TTZ and how most organizations that I have
seen implement their DMZs. 

---
Statement #3
Regarding another problem with hosts in the DMZ:
"Because of the constant exposure of these assets to
the outside world, they must be protected by a greater
investment in security devices, rather than treated as
untrusted, even sacrificial hosts."
---

I just called a couple Fortune 50 and some smaller
customers of ours to ask if their assets in their DMZs
are sacrificial hosts. They said no. 

---
Statement #4
"By 2005, 90 percent of Global 200 gateway firewalls
will do 100 percent deep packet inspection, enabling
them to block application attacks."
---

Now this statement is onto something! We'll get back
to this in just a minute.

---
Statement #5
"IDSs were proposed as the suspenders in the
'belt-and-suspenders' approach to perimeter defense."
---

In this short sentence there are two significant
errors.
 
One- 
IDS is NOT the "belt-and-suspenders" to perimeter
defense, although the mental image is quite
entertaining. Quite the contrast, they are the
"checks-and-balances" to defense technologies. They do
NOT "support" protection, they "detect" when
*protection* mechanisms are failing. They also help to
create audit data useful for the final part of the
security cycle - "reaction."

Two - 
IDSs are not designed just for the perimeter. Many
organizations place them throughout the network, at
server farms, groups with large IP caches or other
data, and even in partner locations. I made a comment
at the top of this which stated, "If you have an IDS
that is telling you that too much is 'suspicious,'
then tune it!" This is the reason why. Not only is
each environment different, but different traffic is
seen in different locations within the same
environments. We do our best to compensate, but only
YOU know YOUR environment best. 

---
Statement #6
"State awareness will enable network agents to scale
to the multigigabit speeds needed."
---

This statement shows an obvious and gross
misunderstanding behind the implementation and design
issues in IDS development. A robust state-tracking
implementation can take just as much overhead as a
good pattern matching, protocol decoding, or anomaly
detection implementation. Look at firewalls for proof
of this.

IDS vendors need to find a balance in implementing
these methodologies, without crushing sensor
performance. No single solution (such as state
tracking) is good enough to be used as a single
detection methodology, or to assert it will enable
multigigabit speeds. 

---
Statement #7
IPS needs to do this: "It requires efficient detection
of malicious attacks. Well-designed network agents
should use a combination of signature, protocol
anomaly detection and traffic analysis to minimize
false positives. State awareness will enable network
agents to scale to the multigigabit speeds needed.
They should be in line to allow them to drop
sessions."
---

This statement appears to show that now even Gartner
has succumbed to marketing hype. You would think they
would have based a paper like this on analysis of some
new vulnerabilities, or trending exploit development
over time, or looking deep into the geopolitical
developments and sociological impacts on attackers and
hacking, right? Based on the above "design
requirements," it sounds as if this document was
written to be a marketing glossy for an IPS vendor.

Marketing is my favorite topic!

Here is a statement taken from a leading IDS/IPS
vendor's website. Not a kind-of leading vendor, a VERY
leading. ;)

"...provides broad-based detection, prevention and
response for attacks and misuse that originate from
across a network. ...using a combination of
sophisticated protocol analysis and pattern matching
to interpret network activity it detects known
attacks, previously unknown attacks, and is immune to
tools that attempt to evade pure pattern matching
systems."

So we'd expect this to be a true statement, right?
Here are the results of a test performed several
months ago. (Maybe 6 to 9 months ago now, and I'm sure
the results of the test have made it back to the
vendor and IDS has been corrected by now.) First, the
two most critical vulnerabilities (one for Unix and
one for Windows) were picked from a 4-6 month time
period. This was done to cover the past two years. 

Then, exploits were harvested for those vulns. Only
exploits from the most public websites like
www.packetstormsecurity.com and www.securityfocus.com
were taken. The final exploit chosen was the one in
each category that was the most easy to use and most
destructively robust. This ensured we were testing the
exploits the kids were most likely to use. 

The IDS was also fully configured and updated. The
point was *not* to evade it or make them look silly,
but to see HOW it viewed certain events compared to
other IDSs. There was a problem though. The following
exploits were missed COMPLETELY:

 - IIS 5.0 .asp overflow wrapped inside of
chunked-encoding exploit with port binding shellcode 
http://packetstormsecurity.org/0206-exploits/DDK-IIS.c

For any "Protocol Decoding IDS" this attack should
have triggered all kinds of HTTP alarms, which it did
not.

 - UPnP remote shell-binding exploit 
http://packetstormsecurity.org/0112-exploits/XPloit.c 
This exploit was chosen for two reasons. One is that
it affects all unpatched Windows ME and Windows XP
systems. The second is that it uses shellcode which is
also used in many other windows-based exploits so it
should be easily identifiable.

 - FrontPage 2000 Server Extensions .asp source
disclosure vulnerability
http://packetstormsecurity.org/0008-exploits/srcgrab.pl.txt

This was chosen mainly because of it's prevalence in
security scanners. This is a vulnerability that many
scanners check for since there is a wealth of
information in many .asp scripts.

 - Apache Chunked Encoding Vulnerability
http://packetstormsecurity.org/worms/apache-worm.c 
Not only is Apache the most widely deployed web server
on planet Earth, but this vulnerability was the basis
for MANY different exploits and Internet worms.

 - Compromise: Command prompt and shell on high port
This test was done because realistically you cannot
expect an IDS/IPS to detect EVERY attack out there.
However, you should expect it to detect the most basic
and generic signs of a successful compromise.

Now if you have missed the significance of these test
results, let me paste the statement that same vendor
made about their technology on a main webpage again:

"...provides broad-based detection, prevention and
response for attacks and misuse that originate from
across a network. ...using a combination of
sophisticated protocol analysis and pattern matching
to interpret network activity it detects known
attacks, previously unknown attacks, and is immune to
tools that attempt to evade pure pattern matching
systems."

All of those attacks were 3-24 months old at the time.
All of those listed above were missed entirely - not
even an unrelated false positive was triggered from
the attacks. Many others (not listed here) were only
detected as something else, and not the actual attack.

We'll elaborate a little more on the subject of IDS
vs. IPS in a moment, but I just wanted to make a note
about vendors who claim to have silver-bullet
solutions. 

Also, if IPS was the end-all solution, why do you
think that every market-leading IDS vendor hasn't
adopted it yet? 

---
Summary page of document:
Is a diagram showing something like a firewall that
can do application content inspection and filtering.
---

Now we have two points pending from above that tie
into the summary of this document. One is the
pros/cons of IDS. Ie: those things that would cause a
company of Gartner's stature to release a paper with
the title of this one. 

ALL IDS methodologies have to deal with false
positives. It's the way the technology works. If you
have a device that is going to tell you about any
potentially suspicious activity, that is exactly what
it is going to do. Just because communications might
be suspicious, does not mean it's going to always be
an attack, but at least you have something there to
inform you about it when everything else on the
network fails to protect you.  And the IDSes that are
better with handling false positives are WORSE when it
comes to the category of producing false negatives. If
you are going to tune down on the amount of things you
consider "suspicious" then of course you increase the
chance of tuning down real attacks also. I could show
you a one-for-one relationship why. You even saw a
little of that in the above test case. Now which
prospect is scarier? 

The whole point of an IDS is to take advantage of the
luxury passive analysis affords you. You can be highly
sensitive to anything that looks slightly suspicious.
You can spread your analysis over a time period
spanning several fractions of a second, several
packets, or even several months. You have the ability
to be highly sensitive to as much (or as little) as
you need to be - to find and detect compromise of your
security policy.

Now don't get me wrong, IPS is an awesome technology.
Bill Boyle from Intruvert put it the most elegantly on
the focus-ids list when there was a thread of people
(including myself) bashing IPS. He said something to
the effect of, "[paraphrasing] We're not claiming to
stop everything, but if we can stop some attacks, then
why wouldn't you?" (Sorry about the previous thread
Bill!) 

This was the best point I've ever heard made about
nIPS, but does it entirely *replace* nIDS as Gartner
has stated? Absolutely not! That idea is about as
ridiculous as stating a DMZ is going to be more secure
if you change the name of it. 

There's another point to make along this vein... How
is an IPS going to block attacks that aren't attacks?
I mean, totally valid traffic that is only dangerous
because of a policy misconfiguration? If you refer to
a poll done by zone-h (arguably the most active
defacement mirror on the net), most defacements (if
you can rewrite data on a server, I'd count that as a
hack) are accomplished because of misconfigurations.
Think about that. That's somewhere around 300 attacks
a day (average) that are reported to zone-h because of
misconfigurations. How many more do you think happen
*every day* that aren't reported? 

This doesn't even take into consideration problems
with custom applications -- like web apps -- which
exist in almost every environment. 

In summary of point one:
 - Good security implementations follow the Protection
-> Detection -> Reaction paradigm. 

IP = Protection
ID = Detection
IP does not = Reaction. 

Better locks on the door don't mean you need to give
up your security cameras. 

Point two:
Richard made it himself which is why I can't believe
he went on with the paper. "... gateway firewalls will
do 100 percent deep packet inspection, enabling them
to block application attacks."

An IPS, being in-line, does not have the indulgence of
being able to be highly sensitive to everything an IDS
can. (Just contrasting the two methodologies here.)
Since it is making the decision to pass or not pass
traffic, it has no room for misjudgment. As such, that
places a severe limitations on its ability to find the
things off-line analysis offers. In addition, analysis
is limited to what can be accomplished in fractions of
a second. There is no opportunity for *real* analysis
and correlation. 

To make an IDS into an IPS border-lines a silly idea,
but to go so far as to say that IPS will replace IDS
entirely is absolute ignorance. And we haven't
addressed the issues of politics, availability,
management, etc...
 
An IPS is not an extension of an IDS, it's an
extension of a firewall. And, that does NOT mean a
firewall with an IDS on/next to it. The discussion of
making a firewall an IPS is kind of an entertaining
one. Most people think they have firewalls all figured
out until you start heading down the path of all the
problems they have. It's funny how the solution to all
of a firewall's problems seems to mirror most people's
conception of what an IPS is...

A paper on that topic might be a good read. Greg
Shipley recently brought the point up in a Network
Computing column
(http://www.nwc.com/1411/1411colshipley.html), and I'd
love to see a technical analysis of the points he and
others have raised. If Gartner decides to take on the
task of writing this, I hope it's done in a more
responsible manner than this was. 

That is what upsets me the most about incidents like
this. Because of the long history Gartner has with
industry reporting, their documents carry a lot of
weight for many organizations. Although, this recent
track record of negligence is disturbing to say the
least. 

-gary


Gary Golomb
Senior Research Engineer
Dragon Intrusion Detection Group
Enterasys Networks



==== Follow-on post:

I got an off-list reply saying I was ranting and
missed the point of the report. (I'll leave him
anonymous. And man, no offense at all! Seriously!) For
better or for worse, he's right and some parts of this
definitely are rants. 

Because of Gartner's weight, there are some serious
and negative side effects of them just defining new
terms on the fly, or saying some technology is
more/less useful based on non-technical findings. They
affect some of us more than others, but it does impact
all us when stuff like this is allowed to go by
unchecked. 

One of the biggest impacts (of several) this is going
to have is on non-technical folks. Now every IDS
vendor under the sun will be renaming their products
to Intrusion Prevention/Protection/Response/etc
Systems. What's that going to do for the people that
don't know any better? Marketing is what makes the
world go around, and if we've made any progress in
forcing IDS vendors to hold to their claims, that's
all probably just been thrown out the window. (I could
write another email on this subject alone, but I'm
probably pushing it enough as-is.) With technical
people in one hand and Gartner in the other, I'll give
you one guess who'll win that battle. 

Not only is this going to hurt the public, who's
trying to learn how to effectively implement these
technologies, it's going to hurt products also.
Especially to meet the 2005 forecast [read: deadline]
set forth by Gartner if [when] vendors reallocate R/D
resources to "prevention" advancements as opposed to
evolving and expanding "detection" technologies. It's
nice to think the two methodologies are completely
interchangeable (as Gartner has so liberally done),
but the truth is, they're not. There isn't a person I
know who'd say that Intrusion Detection is fully
mature and doesn't need any more research. Granted, IP
needs more resources dedicated to it also, but there
are other products purpose-built for "protection" that
seem to make better foundations for advancing this
technology. 

Anyways, there's one other point to be made about this
report. As I see it, the blame is not entirely on
Gartner. This report was written based on the
information made available to the author from vendors.
IPS vendors had a more convincing story. Shame on the
vendors still taking a responsible approach to IPS
technologies for not having a stronger, louder, and
more relevant story *and* actively lobbying it to the
Gartner's of the world. You reap what you sow, or
don't sow.... 

-gary
















































__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com

-------------------------------------------------------------------------------
Attend the Black Hat Briefings & Training, July 28 - 31 in Las Vegas, the 
world's premier technical IT security event! 10 tracks, 15 training sessions, 
1,800 delegates from 30 nations including all of the top experts, from CSO's to 
"underground" security specialists.  See for yourself what the buzz is about!  
Early-bird registration ends July 3.  This event will sell out. www.blackhat.com
-------------------------------------------------------------------------------