Re: IDS vs. IPS deployment feedback

[Jason <security () brvenik com>]

I've lurked and watched as some have struggled to put out FUD that is
absurd at best. I feel it appropriate to to respond and I hope there are
no objections to me summarizing and replying in a single mail. WARNING
it is long as a result.

On Mar 30 @ 11:30 it was stated:

"Proxy firewalls make up a small (and shrinking) percentage of the
market of firewalls. And having worked with over 500 different
companies, my experience is that proxy-based firewalls are rarely
deployed in the manner you describe."

While I certainly appreciate the perspective I must humbly disagree. A
proxy based firewall is not only far superior to the stateful firewalls
that exist today, they are aptly suited for the job. A properly
implemented proxy will beat out every stateful firewall in the market
hands down in a security context.

It was later stated in the same paragraph:

"The default deny from unknown or un allowed protocols is almost ALWAYS
turned off because it breaks some important businesses system that was
poorly coded. Furthermore, a proxy validating protocols still cannot
stop a lot of exploits. Plenty of exploits live quite comfortably inside
the RFC-specs for a protocol. And in this case, your proxy-firewall
would do nothing to stop them. "

This is wholly inaccurate as it relates to technology that was available
in the late 90's and even more inaccurate today. Check Point was capable
of blocking CodeRed when it happened if configured properly. This with
Check Point generally being considered a stateful firewall even though
it is far more capable than the designation implies. Every properly
implemented proxy could and does do the same...

It has been my experience, that by and large, the failure is not in the
technology but in the company's choice of technology, choice of product,
available budget, and choice of trusted advisers.

still later in the same mail it is stated:

"Most firewalls have no insight into application-layer content. And most
exploits are application-layer exploits. This isn't just some insane
idea, it's a fact. You can ignore this and tell yourself 10000 times you
don't need no stinkin' IPS, but the cold hard stiff fact is: firewalls
are not sufficient protection for most organizations. "

Which is perhaps accurate of some of the firewalls commonly deployed
today. An example is that a pix will not get you blocking on application
data, this is perfectly acceptable since it should never be deployed
where there is need for that. It is not accurate to state that the IPS
offers any _application_ awareness or state tracking. I am only aware of
two commonly available IPS technologies that can even come close to
ad-hoc application state awareness, without writing code for a
proprietary closed product. Those two IPS technologies are NFR and Snort.

If you want to eliminate NFR and Snort-based products from the
comparison, then you have an IPS which is as ignorant and incapable of
tracking ad-hoc application state as any firewall could be. These IPS
devices are essentially firewalls with default allow policies and fast
regex engines with drop capability hooked in.

On April 5th @ 3:19pm it is stated:

"Dropped packets happen when people try to ram 1000mbps through an IPS
rated at 200Mbps."

Perhaps this is semantics and I am happy to leave it as such but I must
express an opinion.

Any device when pushed beyond its means _will_ come to it's knees. A
proper firewall or IPS should prevent this by design and as a result
there would be no dropped packets, only packets it refused to service.

If a firewall or IPS drops a packet when operating within spec, it
should be because of a device decision as a result of a policy decision
actively made by the administrator of that device. This is completely
different than the IPS refusing to accept the packet in the first place.
Lets not confuse preservation with performance as advertised.

It is later stated in this mail

"Its impossible to defend 100% against the unknown. You HAVE to make
some type of educated guesses as what is PROBABLE and defend against
that which is MOST PROBABLE. And that is exactly what and IPS does. It
can look at a stream and say: "its HIGHLY unlikely that this gargantuan
binary package in the middle of a ISAPI call is normal, so I am going to
block it."

Or it could be a dcom transfer of brain scans by a custom application
designed to enable remote diagnosis of tumors...

While we agree that it is impossible to have 100% perfection and defend
against everything. We do not agree that an IPS that thinks things are
unlikely and should not occur is the right approach. If I had a dollar
for every "unlikely" thing I've seen happen on a network I would
probably own Microsoft.

and then later:

"When you clear away the hype and FUD, the value of an IPS obvious. You
can lower risk by knowing that set number of vulnerabilities are
blocked, thus reducing the number of incidents that need to be
investigated. "

This of course assumes that the IPS has an opportunity to block the
exploitation at a chokepoint. An IPS is the perfect tool for preventing
attacks you already knew about as they traverse a chokepoint in the
network, its value is clearly obvious in that regard. This does not
translate into reducing risk. This translates into reducing expense as a
result of having to respond to the successful exercising of that risk.

On the 7th of Arpil it is then stated:

"Your 500 number is wrong. When you get into the leading commercial IPSs
(TippingPoint, ISS, Juniper, McAfee) these products on average have
2000-3000 signatures. However, in some technologies, one signature
handles an entire class of vulnerabilities. Where Snort needs multiple
signatures for the same vulnerability, ISS can protect against the
vulnerability with 1 signature. TP is the same. I don't know Juniper and
McAfee as well, but I suspect they are similar. "

I must ask, how is it we are to know exactly how many signatures ISS has
to detect something? That there is but one GUI line item does not mean
there are not 200 supporting checks behind it. Same goes for
TippingPoint, Juniper, McAfee... If it is not open and readily available
for independent review it is but the word of the vendor. How do you even
know what the actual triggering conditions are?

Why is it important to know exactly what the conditions are? The answer
is pretty simple. Without the real information and data we have to do an
actual forensics analysis to determine if something was compromised.
This is because we have no context or meaningful audit of what really
happened when using these closed systems.

In the non attack category we get into the case where someone calls you
and says "My widget maker doesn't work any more" and you then have to go
look for the IP involved and tell them you have no events for it.
Fortunately, the networking team placed a sniffer on the link to
diagnose it for you and it turns out that the IPS device was blocking
some multicast state synchronization and you didn't even have a packet
or the real detection method used to compare and refine for the target
network.

and then in that same mail it is stated:

"Snort also has a lot of unique signatures that people have designed for
highly specialized purposes. That is definitely a benefit to some
organizations. But, those signatures are only useful in those unique
situations. And all the commercial products support custom signatures -
so you can do the same thing for your TP or ISS box."

Interesting opinion. Try to write a set of detection signatures for
those closed products that will track a valid login to a custom
application and then the exploitation of an vulnerability. You cannot do
it because they do not allow the tracking of application state in the
signature engines. The ability to create a regex in these products is
hardly the same capability you get in an advanced rules engine like
Snort. Try writing a signature for some custom protocol that rides over
TCP and looks a lot like telnet but is not and does not conform to an
RFC in the slightest.

It is later stated in that same mail:

"Furthermore, Snort rules are developed by volunteers (or Sourcefire).
As such, SNORT is usually behind the curve on new signatures. ISS, for
example, does their own independent security research an has signatures
to protect against things that Snort people don't even know about. Other
vendors buy exploits from the hacker market - again giving them access
to vulnerabilities long before it hits the public and subsequently the
people who develop SNORT signatures. "

Lets clarify that a little and please do not confuse professionally
developed detection with the rules put out by the hobbyists in the
market. Snort rules can be written by anyone and as a result there are
many organizations and people that write rules. The premier organization
that writes rules is Sourcefire. Sourcefire purchases information from
the various vulnerability information warehouses like every other vendor
does and has a dedicated research team that is toe to toe if not better
than the other groups out there. That Sourcefire is the premier writer
of rules is no strange coincidence since Sourcefire also writes Snort
and focuses on detection of the exploitation of _vulnerabilities_ not
the transmission of _exploits_

Sourcefire is most definitely ahead of the curve and has detection out
before the threat. Snort by itself may be behind the curve if you choose
not to subscribe to the Sourcefire VRT updates and instead decide to
play the risk game. This is a decision that people are free to make and
often do. This is not a fault of Snort, the technology, community, or
companies that produce professional high quality rules for Snort. This
is a failure of the administrator and nothing you do with technology
will eliminate that failure.

It is later stated in this same mail:

"The 90% thing you're coming up with is just false. You're assuming that
all those signatures represent a serious attack. And you're also
assuming that quantity of signatures is the measure of effectiveness."

Is there an attack that is not serious? Is there a policy violation that
is not serious? Have you looked at the scope and depth of the rules
available for Snort? This statement is simply baseless FUD.

and then it is stated:

"A poorly maintained, tuned or implemented Snort sensor is just as
useless as a poorly maintained, tuned, or implemented ISS sensor."

We certainly agree here. Without maintenance all products are useless.

On April 7th at 12:05pm it is stated:

"I maintain my position that most businesses lack the analytical
capabilities to deploy resource intensive technologies (like SNORT).
Hence, commercial IPS that can filter off a set of known vulnerabilities
reduces the overall workload and offers a layer of protection."

Snort can just as easily filter off these attacks. When backed by a
capable company like Sourcefire the resource intensive challenge is also
significantly reduced. Automatic updates can even be configured to
maintain detection capabilities without ever touching the system.
Combine the detection with contextual awareness of RNA and you get
impact assessments that prioritize your efforts. Snort is but a piece of
the larger security puzzle and anyone that spends this much time trying
to discredit the technology is clearly feeling the pain of Snort.

It is then stated:

"Also, the majority of attacks in the wild are well-known and easily
detected and blocked."

This is completely subjective and I wish people would stop trying to
push an opinion on the market. What is a risk to one company may well be
a nuisance to another.

Looking back on some of the previous mails in this thread I have to ask.
Would any of those attacks happen to be considered one that is not serious?

And finally on April 10th at 3:54pm it is stated:

"Yes...SOURCEFIRE customers get those signatures early. They get handed
out to the Snort world well after the fact. SourceFire is a commercial
company and you must PAY to get their product."

Firstly, "Well after the fact" is only 5 days. You can also subscribe to
get the rules for Snort in real-time. I don't see how having to pay
matters since the alternatives are all commercial as well. If you have
the need for protection and not the budget then you can run Snort and
become a subscriber.

"In other words - Sourcefire is no different than TP, ISS or any other
commercial vendor in this regard. As such, we're all just selling what
we know."

Being a Sourcefire employee, major contributor to Snort, and general
gear head that likes to beat on these things I can definitely say that
Sourcefire and Snort are absolutely different than ISS or TP or any
other commercial vendor.

The next time anyone wants to get on the bandwagon and start throwing
FUD around I would like them to take a moment to realize that Snort is
deployed and runs in more networks than all of the other commercial
vendors combined.

Andrew Plato wrote:
> Yes...SOURCEFIRE customers get those signatures early. They get handed
> out to the Snort world well after the fact. SourceFire is a commercial
> company and you must PAY to get their product. 
>
> In other words - Sourcefire is no different than TP, ISS or any other
> commercial vendor in this regard. As such, we're all just selling what
> we know. 
>
> ___________________________________
> Andrew Plato, CISSP
> President/Principal Consultant
> Anitian Enterprise Security
>
>
>
> -----Original Message-----
> From: Richard Bejtlich [mailto:taosecurity () gmail com] 
> Sent: Monday, April 10, 2006 10:36 AM
> To: Andrew Plato
> Cc: Basgen, Brian; focus-ids () securityfocus com
> Subject: Re: IDS vs. IPS deployment feedback
>
> You are not familiar with modern Snort signature development by the
> Sourcefire Vulnerability Research Team. See:
>
> http://www.sourcefire.com/services/sf_vrt.html
>
> For one example:
>
> http://www.sourcefire.com/news/press_releases/pr121504.html
> _________________________________________________
> NOTICE:
> This email may contain confidential information, 
> and is for the sole use of the intended recipient.  
> If you are not the intended recipient, please reply 
> to the message and inform the sender of the error 
> and delete the email and any attachments from 
> your computer. 
> _________________________________________________
>
>
> ------------------------------------------------------------------------
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it 
> with real-world attacks from CORE IMPACT.
> Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
> to learn more.
> ------------------------------------------------------------------------

------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it 
with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
------------------------------------------------------------------------