Re: Vulnerability statistics

[Adam Shostack <adam () homeport org>]

Interesting work!

There's a couple of biases here:

1) Only 'widely deployed' software gets into the CVE.  Thus, a bug in
say, Hotmail or Google wouldn't make it in, because it's unique.

2) The CVE entries don't give you a scope for each vuln, based on how
widespread it is.  The CERT Vuln metric includes that information, but
it (intentionally) conflates severity with how widespread the target
is.  Thus, an IE vuln that lets you crash the system would likely get
a higher metric than a Galeon vuln that lets you run
code. http://www.kb.cert.org/vuls/html/fieldhelp#metric

This lack of good information about what really causes security
problems makes it hard to do good security work that will help lots of
people:  Where do you start?  I think this is the most pernicious
aspect of current attitudes towards disclosure.

Get a bunch of security experts in a room with a bottle of scotch, and
we've all been hacked.  Attack is easier than defense.  But we're
hesitant to admit to the effect, which is we all get 0wned now and
again.

Adam

On Fri, Jan 07, 2005 at 11:18:41AM -0800, Michael Howard wrote:
| I wrote some code to pull down the CVE XML file from cve.mitre.com and
| parse the results looking for keywords. This is NOT scientific, but
| here's my results:
| 
| Getting stats for 2004
| TotalCount      1339
| isReserved      204
| isRejected      15
| isUnknown       50
| 
| isBO    296
| isFormatString  33
| isIntOverflow   53
| isSQLinjection  30
| isXSS   73
| isInjection     60
| isTooMuchTrust  119
| isSymlink       49
| isRace  8
| isWeakPermission        13
| 
| I have yet to analyze the other bugs not in the list above - some of the
| bug texts are very vague...
| 
| [Writing Secure Code] http://www.microsoft.com/mspress/books/5957.asp
| [Protect Your PC] http://www.microsoft.com/protect
| [Blog] http://blogs.msdn.com/michael_howard
| 
| [On-line Security Training]
| http://mste/training/offerings.asp?TrainingID=53074
| 
| 
| -----Original Message-----
| From: Benjamin Livshits [mailto:livshits () cs stanford edu] 
| Sent: Thursday, January 06, 2005 1:56 PM
| To: webappsec () securityfocus com
| Subject: Vulnerability statistics
| 
| Looking at the OWASP's top ten list, are there any recent studies as to
| what fraction of vulnerabilities accounts for each of the top ten
| categories?
| 
| What about the percentage of vulnerabilities caused by coding errors vs
| configuration flaws?
| 
| Thanks,
| -Ben
|