Re: BSIMM3 lives

[Gary McGraw <gem () cigital com>]

hi steve and sc-l,

Sorry for the delay in responding.  I am just catching up after spending
last week in Bloomington, Indiana.  Some quick answers:

> 1) Was any analysis done to ensure that the 3 levels are consistent
> from a maturity perspective - for example, if an organization
> performed an activity at level 2, that there was a high chance that
> it also performed many of the level-1 activities?  For example,
> many T2.x activities were done by more organizations than their
> counterpart T1.x activities, and there's a similar pattern with
> some SR2.x versus SR1.x.

We have done that kind of analysis twice...once between BSIMM and BSIMM2
and again between BSIMM2 and BSIMM3.  Our main objective is to ensure that
the levels we have identified hold statistical water in the entire
population.  We are less concerned with particular threads or associated
activity chains at this point.  The (10) minor adjustments to the BSIMM
that we have made over the years were driven by data analysis.

> 2) Any thoughts on why the financial services vertical scored
> noticeably lower than ISVs on Code Review, Architectural Analysis,
> etc.?  Maybe ISVs have a better "infrastructure" for launching
> these activities because code development is a core aspect of
> their business?

Good question.  The main thing we noticed between BSIMM2 and BSIMM3 is
that ISVs began to pull away from FIs as a population in terms of
maturity.  This is most likely due to different approaches to the
recession.  The FIs pulled back from all operations and restructured staff
(often with layoffs).  The ISVs reinvested their profits into themselves
(and slowed the M&A engine down a gear or two).  Some of the ISV
investment went directly into the SSG.

Back in BSIMM2, there was only a 9 activity difference between ISVs and
FIs as populations given a T test (see Figure 4 of BSIMM2: Measuring the
Emergence of a Software Security Community
<http://www.informit.com/articles/article.aspx?p=1592389> (May 12, 2010)).
 I am sure this has widened.

WRT technical activities such as code review and architecture analysis,
your theory is a good one.  I would enhance it by pointing out that
process-oriented activities are often an easier thing for FIs to establish
than for ISVs.  These two factors together are likely to account for the
difference.

> 3) The wording about OWASP ESAPI in SFD2.1 is unclear: "Generic open
> source software security architectures including OWASP ESAPI should
> not be considered secure out of the box."  Does Struts, mentioned
> earlier in the paragraph, also fall under the category of "not
> secure out of the box?"  Are you saying that developers must be
> careful in adopting security middleware?

Of course struts is not secure out of the box, which is the whole point of
the activity.  The major difference between struts as insecure and ESAPI
as insecure is that ESAPI claims to be a secure solution, though it is
often not.  One might argue that it is worse to claim to be secure and not
to be than to ignore the whole thing, but that's not really worth
pursuing.  For more regarding Cigital's view on ESAPI, see
http://www.cigital.com/justiceleague/2011/09/24/suggestions-for-esapi-2-1-a
nd-beyond/ 

Thanks for your questions.  Hope these answers help.

gem

P.S. Cross posting to the BSIMM list.


company www.cigital.com
podcast www.cigital.com/silverbullet
blog www.cigital.com/justiceleague
book www.swsec.com










On 10/15/11 5:45 PM, "Steven M. Christey" <coley () rcf-smtp mitre org> wrote:

> Gary,
>
> Congratulations to you, Brian, Sammy, and the rest of the BSIMM3
> community!
>
> I have a few questions:
>
> 1) Was any analysis done to ensure that the 3 levels are consistent
>    from a maturity perspective - for example, if an organization
>    performed an activity at level 2, that there was a high chance that
>    it also performed many of the level-1 activities?  For example,
>    many T2.x activities were done by more organizations than their
>    counterpart T1.x activities, and there's a similar pattern with
>    some SR2.x versus SR1.x.
>
> 2) Any thoughts on why the financial services vertical scored
>    noticeably lower than ISVs on Code Review, Architectural Analysis,
>    etc.?  Maybe ISVs have a better "infrastructure" for launching
>    these activities because code development is a core aspect of
>    their business?
>
> 3) The wording about OWASP ESAPI in SFD2.1 is unclear: "Generic open
>    source software security architectures including OWASP ESAPI should
>    not be considered secure out of the box."  Does Struts, mentioned
>    earlier in the paragraph, also fall under the category of "not
>    secure out of the box?"  Are you saying that developers must be
>    careful in adopting security middleware?
>
>
> - Steve


_______________________________________________
Secure Coding mailing list (SC-L) SC-L () securecoding org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates
_______________________________________________