Re: BSIMM3 lives

[Gary McGraw <gem () cigital com>]

hi chris,

Thanks for posting your data.  This is great.

The forty-two participating organizations in BSIMM3 are drawn from eight
verticals (with some overlap): financial services (17), independent
software vendors (15), technology firms (10), telecommunications (3),
insurance (2), energy (2), media (2), and healthcare (1). Those companies
among the forty-two who graciously agreed to be identified include: Adobe,
Aon, Bank of America, Capital One, The Depository Trust & Clearing
Corporation (DTCC), EMC, Fannie Mae, Fidelity, Google, Intel, Intuit,
Mashery, McKesson, Microsoft, Nokia, QUALCOMM, Sallie Mae, SAP, Scripps
Networks Interactive, Sony Ericsson, Standard Life, SWIFT, Symantec,
Telecom Italia, Thomson Reuters, Visa, VMware, Wells Fargo, and Zynga.

ISVs: Adobe, EMC, Google, Intuit, Mashery, McKesson, Microsoft, SAP, Sony
Ericson, Symantec, Vmware, Zynga (and 3 un-named firms).

Though our ISV population is certainly biased towards big companies, there
are a couple of smaller firms in the mix (and even some very small ones).

I basically agree with Steve's theory on why code review, architecture
analysis, and security testing are more commonly observed in ISVs (with
the added comment regarding FIs and process).

gem

P.S.  Cross posting to the BSIMM list again.

company www.cigital.com
podcast www.cigital.com/silverbullet
blog www.cigital.com/justiceleague
book www.swsec.com

On 10/17/11 11:49 AM, "Chris Wysopal" <cwysopal () Veracode com> wrote:

> Steve,
>
> Here is some information relative to your inquiry point #2 below.
>
> Veracode's data from our State of Software Security Report Vol 3 found
> that internally developed financial services software had less and lower
> severity defects than ISVs.  ISVs actually had the worst security through
> the lens of our automated static and dynamic analysis. Chart is attached.
> Here are the numbers:
>
> Application Performance by Industry (% acceptable level of security)
>
> Overall                42%
> Financial      47%
> Government     50%
> Software       34%
> Other          40%
>
> Sample size is 4835 applications total.
>
> One theory on the discrepancy between what we are seeing and BSIMM is our
> sample of ISVs spans the entire industry by revenue.  We had many
> companies in each of the following revenue categories: Under $50M,
> $50-500M, $500M-1B, and over $1B.  Looking at the BSIMM ISV participants,
> the companies all look like they are in the over $1B category.
>
> But to add a contradicting data point, we found that % acceptable level
> of security didn't change much based on ISV revenue. Whisker chart
> attached.  I pulled out this data in my argument against Mary Ann
> Davidson when she posited that maybe small ISVs can't be trusted to get
> software security right but big companies can.
>
> -Chris
>
>
> -----Original Message-----
> From: sc-l-bounces () securecoding org
> [mailto:sc-l-bounces () securecoding org] On Behalf Of Steven M. Christey
> Sent: Saturday, October 15, 2011 5:45 PM
> To: Gary McGraw
> Cc: Secure Code Mailing List
> Subject: Re: [SC-L] BSIMM3 lives
>
>
> Gary,
>
> Congratulations to you, Brian, Sammy, and the rest of the BSIMM3
> community!
>
> I have a few questions:
>
> 1) Was any analysis done to ensure that the 3 levels are consistent
>    from a maturity perspective - for example, if an organization
>    performed an activity at level 2, that there was a high chance that
>    it also performed many of the level-1 activities?  For example,
>    many T2.x activities were done by more organizations than their
>    counterpart T1.x activities, and there's a similar pattern with
>    some SR2.x versus SR1.x.
>
> 2) Any thoughts on why the financial services vertical scored
>    noticeably lower than ISVs on Code Review, Architectural Analysis,
>    etc.?  Maybe ISVs have a better "infrastructure" for launching
>    these activities because code development is a core aspect of
>    their business?
>
> 3) The wording about OWASP ESAPI in SFD2.1 is unclear: "Generic open
>    source software security architectures including OWASP ESAPI should
>    not be considered secure out of the box."  Does Struts, mentioned
>    earlier in the paragraph, also fall under the category of "not
>    secure out of the box?"  Are you saying that developers must be
>    careful in adopting security middleware?
>
>
> - Steve
> _______________________________________________
> Secure Coding mailing list (SC-L) SC-L () securecoding org List information,
> subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
> List charter available at - http://www.securecoding.org/list/charter.php
> SC-L is hosted and moderated by KRvW Associates, LLC
> (http://www.KRvW.com) as a free, non-commercial service to the software
> security community.
> Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates
> _______________________________________________


_______________________________________________
Secure Coding mailing list (SC-L) SC-L () securecoding org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates
_______________________________________________