BSIMM: Confessions of a Software Security Alchemist (informIT)

[jsteven at cigital.com (John Steven)]

Steve,

You saw my talk at the OWASP assurance day. There was a brief diversion about the number of "business logic" problems 
and "design flaws" (coarsely lumped together in my chart). That 'weight' should indicate that-at least in the subset of 
clients I deal with-flaws aren't getting short-shrift.

http://www.owasp.org/images/9/9e/Maturing_Assessment_through_SA.ppt (for those who didn't see it)

You may also want to look at my OWASP NoVA chapter presentation on "why" we believe Top N lists are bad... It's not so 
much a rant as it is a set of limitations in ONLY taking at Top N approach, and a set of constructive steps forward to 
improve one's practices:

http://www.owasp.org/images/d/df/Moving_Beyond_Top_N_Lists.ppt.zip

I cover how one should cause their own organization-specific Top N list to emerge and how to manage it once it does.

----
John Steven
Senior Director; Advanced Technology Consulting
Direct: (703) 404-5726 Cell: (703) 727-4034
Key fingerprint = 4772 F7F3 1019 4668 62AD  94B0 AE7F EEF4 62D5 F908

Blog: http://www.cigital.com/justiceleague
Papers: http://www.cigital.com/papers/jsteven

http://www.cigital.com
Software Confidence. Achieved.




On 3/18/09 6:14 PM, "Steven M. Christey" <coley at linus.mitre.org> wrote:



On Wed, 18 Mar 2009, Gary McGraw wrote:

> Many of the top N lists we encountered were developed through the
> consistent use of static analysis tools.

Interesting.  Does this mean that their top N lists are less likely to
include design flaws?  (though they would be covered under various other
BSIMM activities).

> After looking at millions of lines of code (sometimes constantly), a
> ***real*** top N list of bugs emerges for an organization.  Eradicating
> number one is an obvious priority.  Training can help.  New number
> one...lather, rinse, repeat.

I believe this is reflected in public CVE data.  Take a look at the bugs
that are being reported for, say, Microsoft or major Linux vendors or most
any product with a long history, and their current number 1's are not the
same as the number 1's of the past.