Re: BSIMM3 lives

[Chris Wysopal <cwysopal () veracode com>]

Steve,

Here is some information relative to your inquiry point #2 below.

Veracode's data from our State of Software Security Report Vol 3 found that internally developed financial services 
software had less and lower severity defects than ISVs.  ISVs actually had the worst security through the lens of our 
automated static and dynamic analysis. Chart is attached. Here are the numbers:

Application Performance by Industry (% acceptable level of security)

Overall         42%
Financial       47%
Government      50%
Software        34%
Other           40%

Sample size is 4835 applications total.

One theory on the discrepancy between what we are seeing and BSIMM is our sample of ISVs spans the entire industry by 
revenue.  We had many companies in each of the following revenue categories: Under $50M, $50-500M, $500M-1B, and over 
$1B.  Looking at the BSIMM ISV participants, the companies all look like they are in the over $1B category. 

But to add a contradicting data point, we found that % acceptable level of security didn't change much based on ISV 
revenue. Whisker chart attached.  I pulled out this data in my argument against Mary Ann Davidson when she posited that 
maybe small ISVs can't be trusted to get software security right but big companies can.

-Chris


-----Original Message-----
From: sc-l-bounces () securecoding org [mailto:sc-l-bounces () securecoding org] On Behalf Of Steven M. Christey
Sent: Saturday, October 15, 2011 5:45 PM
To: Gary McGraw
Cc: Secure Code Mailing List
Subject: Re: [SC-L] BSIMM3 lives


Gary,

Congratulations to you, Brian, Sammy, and the rest of the BSIMM3 community!

I have a few questions:

1) Was any analysis done to ensure that the 3 levels are consistent
    from a maturity perspective - for example, if an organization
    performed an activity at level 2, that there was a high chance that
    it also performed many of the level-1 activities?  For example,
    many T2.x activities were done by more organizations than their
    counterpart T1.x activities, and there's a similar pattern with
    some SR2.x versus SR1.x.

2) Any thoughts on why the financial services vertical scored
    noticeably lower than ISVs on Code Review, Architectural Analysis,
    etc.?  Maybe ISVs have a better "infrastructure" for launching
    these activities because code development is a core aspect of
    their business?

3) The wording about OWASP ESAPI in SFD2.1 is unclear: "Generic open
    source software security architectures including OWASP ESAPI should
    not be considered secure out of the box."  Does Struts, mentioned
    earlier in the paragraph, also fall under the category of "not
    secure out of the box?"  Are you saying that developers must be
    careful in adopting security middleware?


- Steve
_______________________________________________
Secure Coding mailing list (SC-L) SC-L () securecoding org List information, subscriptions, etc - 
http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the 
software security community.
Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates
_______________________________________________

_______________________________________________
Secure Coding mailing list (SC-L) SC-L () securecoding org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates
_______________________________________________