Re: FW: BlackIce Defender??? (and CVE again)

[Adam Shostack <adam () homeport org>]

On Wed, Oct 27, 1999 at 09:05:36AM -0500, Rick Smith wrote:
| I have an *old* thing sitting around for some product that "detects over
| 270" signatures. The Black Ice stuff I saw claimed around 200. Of course,
| heaven knows what they're really counting.
| 
| This segues rather nicely into the Common Vulnerability Enumeration
| discussion -- CVEs may turn into the marketing touchstone: "we detect
| everything in the CVE." It's essentially a replay of anti-virus
| competition, but I don't think anyone ever came up with a third party
| enumeration of viruses.

Its really hard to catch everything in the CVE, since it includes
things like ssh agent credential stealing (CVE-1999-0013) and remote
buffer overflows in mountd (CVE-1999-0002).  You're going to need a
widely deployed, very broadly cross platform, OS and network ID system
to do it.  On the bright side, its all misuse detection, no anomaly
detection, so that narrows the scope a little.

One of the things that I personally hope to see the CVE used for is
for customers (or organizations like SANS, or a test lab) to be able
to map between products and say that Product A's 38 checks actually
catch the same set of potential problems that Product B looks for
using 210 checks, and that A detects 8 CVE-listed
vulnerabilities that B is missing, while B has these 31 that
Julliet is missing.  (Once we get there, we can talk about catching
reliably..)

As an aside, actually doing this analysis across three or four
products without the CVE is really, really hard, and you end up
guessing a lot about what each product is actually testing based on
the descriptions, and based on packet dumps.  However, since most
scanners use some level of inference to detect things that they
report, the packet dumps aren't always all that useful.  So, its
nearly impossible to do an honest assessment of what each product
catches.  (I want an honest assessment internally to know what I need
to add to my product since there is no database to check against.)
When there is no honest assessment, you're forced to get into the
numbers game to which Rick alludes.

Adam


-- 
"It is seldom that liberty of any kind is lost all at once."
                                                       -Hume