The Organic Secure SDLC

[John Steven <jsteven () cigital com>]

Paco,

Thank you for cogently clarifying BSIMM. I'm a bit disappointed in the community's ignorance regarding the model given 
it's both freely available and Creative Commons licensed. Equally disappointing, to me, are positions borne out of a 
"Just use [MyModel™: BSIMM || SAMM]" perspective. Rohit asked:

> If you're an actual practitioner who has lived through developing a secure SDLC I'd love to hear your thoughts about 
> the model's accuracy / relevancy.

Responses to this request would provide this mailing list's readership more value. As one practitioner responsible for 
several SDL programs, I'll respond ignoring Organic vs. BSIMM. I don't see much value in such a comparison.

[Is 'Organic' a model?]

Yes. Paraphrasing one definition, a model is anything that abstracts a system's factors in a way to helps its users 
quickly gain insight into the subject's behavior.

Inaccuracy isn't a fatal blow to a model, quoting Paul Wilmott's Manifesto [BW1]:

        • I will remember that I didn't make the world and that
          it doesn't satisfy my equations.

        • Though I will use models boldly to estimate value, I 
          will not be overly impressed by mathematics.

        • I will never sacrifice reality for elegance without 
          explaining why I have done so. Nor will I give the 
          people who use my model false comfort about its 
          accuracy. Instead, I will make explicit its 
          assumptions and oversights.

        • I understand that my work may have enormous effects 
          on society and the economy, many of them beyond my 
          comprehension.

...Navigators managed rather well with a "Flat Earth" hypothesis for some time--no? So, we don't need over one hundred 
activities in our app. sec. model in order to provide value.   

[Motivation]
Most of you know I respect Rohit a fair amount and so when I read his post, you can imagine my thought, "In a world 
aware of BSIMM what is the value of 'Organic'?" with honest curiosity, not disdain. I immediately guessed 'Organic' was 
meant to address a common complaint regarding almost every prior model: 

        "I'm challenged applying <this> to smaller shops just 
         beginning their Application Security initiatives"

Jim Bird has thoughtfully discussed the "one man shop" problem extensively in his blog [JB1]. Rohit's own explanation 
mentions "no top down support" as an indication of model applicability.

[Accuracy]
'Organic' ignores a lot of key components that even smaller shops already have in place or care about improving. Three 
essential ones include 1) measurement and iterative approach [JB2], 2) security policy [PC1], and 3) security 
toolkits/frameworks [JB2][FM1]. While Rohit's post indicates explicitly that things have been omitted, he focuses on 
having left out architecture and related activities.

To me, even if 'Organic' is designed to focus only on development activities, ignoring a potential need for compliance 
to regulatory/security policy, leveraging toolkits to make developers' jobs easier, or failing to set up a 
measure-and-iterate loop are dire mistakes. I can point to small organizations that have taken very different tacks and 
don't fit the model. Some start with training. Others lean on SCR tools or security toolkits before ever 
institutionalizing pen-testing. 

Perhaps it's inaccurate. Maybe it doesn't meet our industry need for addressing "one man shop". So is it 
good-for-nothing? No. It's useful.

[Value]
An immediate value that jumped out at me was the "Climb the Wall" element [BS1]. I can again pull from experience 
organizations that don't fit the model but the single most salient 'take home' from 'Organic', to me, is:

        You're going to need a champion, and until you "climb 
        the wall" you may hit barriers to progress.

This concept seems to resonate with almost every security professional, even if the form in which it comes can be 
contentious. 

Yet, often as vendors, internal champions, or <whatever> we don't always make this requirement for progress explicit. 
We often underestimate its cost, LoE, and impact. As a result things stall or slide backwards.

Another key model elements that resonate with me is "Have a security champion":

> It's also worth pointing out that while the whole process grows organically, at some stage it still requires a 
> champion within the organization to get the organization to move to the next step. That champion usually has 
> application security in their job description, but not always.

BSIMM respondents handle this in consistent fashion [BS2] but Rohit's way of explaining this is perhaps less foreboding 
to those getting their start and anointing their first victim <cough> champion.

There's also tremendous value (to the 'Organic' model) in admitting what I think we all know implicitly: programs may 
have to "show need" (through assessment) in order to progress. It's not shocking to see penetration testing at the 
beginning of 'Organic'--" 'sploits always create splash" as I say. 

[Conclusion]
To me, the 'Organic' model suffers from key inaccuracies due to omission. As such, it doesn't particularly address the 
principal criticism of existing models. Its value stems from simplicity and an potentially clear way to drive its users 
through key tenets (my summary, not Rohit's):

A) Anoint a champion
B) Show need
C) Educate execs
D) Drive assessment earlier in lifecycle
E) Bake assessment into BAU (QA)

The ability to say these things succinctly to an organization starting its Application Security journey provides value. 
I do not, however, believe that we're going to see security assessment applied by business unit/product team QA folk in 
a BAU scenario in the next 3-5 years, with the notable exception of organizations like Microsoft. 

If 'Organic' was mine, I would attempt to amplify its positives by converting it from a SDL Model to a method 
accompanied by case studies. I'd drive it towards showing, backed by case study, how "climbing the wall" can be 
accomplished most effectively. This is a tough nut and one worth cracking.

-jOHN
----
John Steven
Senior Director; Advanced Technology Consulting
Desk: 703.404.9293 x1204 Cell: 703.727.4034
Key fingerprint = 4772 F7F3 1019 4668 62AD  94B0 AE7F EEF4 62D5 F908

twitter: @m1splacedsoul
Rss: http://feeds.feedburner.com/M1splacedOnTheWeb
web: http://www.cigital.com
Software Confidence. Achieved.


* [BW1] http://www.businessweek.com/magazine/content/09_02/b4115059823953_page_2.htm
* [JB1] http://swreflections.blogspot.com/2009/04/opensamm-shows-way.html
* [JB2] http://swreflections.blogspot.com/2011/06/safer-software-through-secure.html
* [PC1] https://www.pcisecuritystandards.org/security_standards/
* [FM1] http://shiro.apache.org/
* [BS1] Analogous to BSIMM SM1.3 "Educate Executives":  http://bsimm.com/online/governance/sm/?s=sm1.3#sm1.3
* [BS2] http://www.informit.com/articles/article.aspx?p=1434903

On Jul 19, 2011, at 11:24 AM, Paco Hope wrote:

> Jim,
>
> You're spot on. BSIMM is not a lifecycle for any company. Heck, it's not even a set of recommendations. It's simply a 
> way to measure what a firm does. It's a model formulated from observations about how some firms' implement software 
> security in their lifecycles. You'll never catch us calling the BSIMM a lifecycle.
>
> As for not translating into the SMB market, I don't understand that. Unlike, say prescriptive standards which say 
> "thou shalt do X" regardless of how big you are, the BSIMM measures maturity of what a firm actually does. There is 
> no reason an SMB could not measure the maturity of their effort using the BSIMM.
>
> Maturity is not a function of size. A team of 10 developers might score higher on various criteria than a 
> multi-national bank that has a whole team of people dedicated to app sec. Maturity is a function of the depth to 
> which one takes a certain activity and their capability within that activity.
>
> This isn't Pac-Man, either. The goal is not to get the highest score and an extra man. :) The goal is to put the 
> right level of effort into the right places. A firm can't do that until they know how much effort they're spending on 
> different activities. The BSIMM will illuminate the level of effort. It allows a firm to decide to rebalance and 
> spread the budget/people around across the activities that make sense. Whether that's a team of 10 developers or a 
> team of 1000 developers, the principle is the same. The execution varies.
>
> Here's another analogy. You can have a GPS and know your exact coordinates, to within 3 meters, but not know how to 
> get to the airport by car. The BSIMM will tell you your coordinates at the present time. It does not tell you the 
> best way to the airport. It can tell you the crow-fly distance to the airport, but it can't tell you that the airport 
> is where you want to be.
>
> Paco
>
>
> Paco,
>
> By your same logic I would not consider BSIMM a lifecycle either. It's
> a thermometer to measure an SDLC against what some some of the largest
> companies are doing. As others have noted, BSIMM  does not translate
> well into the SMB market where most software is written. Don't get me
> wrong, BSIMM is very interesting data and is useful. But a
> comprehensive secure software lifecycle for every company it is not.
>
> - Jim Manico

Attachment: smime.p7s
Description:

_______________________________________________
Secure Coding mailing list (SC-L) SC-L () securecoding org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates
_______________________________________________