Secure Coding mailing list archives
Re: The Organic Secure SDLC
From: Rohit Sethi <rklists () gmail com>
Date: Thu, 11 Aug 2011 15:36:31 -0400
Hi Jim, Jim, thanks for the comments. It's a fair statement that pen tests don't just happen. There are many organizations who don't pay attention to application security at all - and they don't really fit in this model. You're bang on about the lack of design activities. There just doesn't seem to be consistency here until people take a more meaningful approach like the SDL. That's not to say there aren't exceptions - like we mentioned in the posting, many organizations *do* have some sort of design or architecture assessment it just doesn't appear to be consistent in our observations. With respect to implementing metrics, I think this is a sign of maturity that means organizations are pulling away from a reactive approach. To keep the model simple, we've left out details about iterating although it's very important. Tool selection would typically be contained within the individual step to which the tool applies (e.g. static analysis within source code review). On Thu, Aug 11, 2011 at 1:58 PM, Jim Bird <jimbird () shaw ca> wrote:
Hi Rohit, I just returned from overseas and read through the original post and this email thread. If this Organic model is descriptive (based on what you've observed at companies that you've done work for) then this progression seems to make sense for companies who are working on a reactive basis, and starting with outside help. I guess that it starts with consultant-based work like pen testing and source code review, because the customers that call in consultants would ask for this. Of course then a prerequisite would be some kind of business case or risk assessment or other trigger (attack, CEO reading scary things in the Wall Street Journal, ...) to bring in consultants in the first place to see just how bad things are. Pen tests don't just happen. As John Steven pointed out, there are other important steps like putting in metrics and tracking, and implementing/upgrading tools/frameworks, and (in my experience at least, an important early step) understanding (and later tracking changes to) the attack surface. And iterating through all of this. I can see how Cigital's experience with larger enterprise customers that drives BSIMM would be different, because these customers themselves would drive additional requirements and have additional resources at hand, and because Cigital has its own methods and engagement model and practices and tools that it would bring into the customer. I am surprised to see that this model is so "code heavy/design lite": there's little emphasis on threat modeling / ARA maybe because many companies find it so hard to do? I like the idea of an end-state where security gets burned in to QA like other problems in software development, making the team responsible for security in reviews and testing etc. That's a big step to get to. /Jim ----- Original Message ----- From: Rohit Sethi <rklists () gmail com> Date: Tuesday, July 19, 2011 4:18 pm Subject: Re: The Organic Secure SDLC To: John Steven <jsteven () cigital com> Cc: Secure Code Mailing List <sc-l () securecoding org>, "jimbird () shaw ca" < jimbird () shaw ca>, Paco Hope <Paco () cigital com>Hi John, Thanks for the feedback. This is exactly what we were looking for. We've certainly sought simplicity in this model, even at the expense being incomplete. It's not necessarily aimed at the one man shop - it's aimed at any organization where secure software is just not an explicit top-level priority. It doesn't address any of the short-comings of any previous model because it's not an alternative to them. It's simply an observation of a seemingly natural - organic- progression of steps. I agree with you about its value. No organization matches this model completely - there are often additional steps, some that you mentioned, which one organization or another takes or where the order is slightly different than what we've outlined. You can think of the steps we've outlined as a line of best fit: the steps we've seen to be most common. I'm often surprised to find security practitioners thinking they are way behind industry because they are are struggling to convince the lines of business to participate in security activities. One motivation for the model is to let those practitioners know that they're not alone. Case studies are a fantastic idea. We will add these to the model over time. We also want to be able to point to useful resources for people at each step, so if you (i.e. anyone reading this) has written relevant articles or whitepapers let me know. On Tue, Jul 19, 2011 at 4:43 PM, John Steven <jsteven () cigital com> wrote:Paco, Thank you for cogently clarifying BSIMM. I'm a bitdisappointed in thecommunity's ignorance regarding the model given it's bothfreely availableand Creative Commons licensed. Equally disappointing, to me,are positionsborne out of a "Just use [MyModel™: BSIMM || SAMM]"perspective. Rohitasked:If you're an actual practitioner who has lived throughdeveloping asecure SDLC I'd love to hear your thoughts about the model'saccuracy /relevancy. Responses to this request would provide this mailing list'sreadership morevalue. As one practitioner responsible for several SDLprograms, I'llrespond ignoring Organic vs. BSIMM. I don't see much value insuch acomparison. [Is 'Organic' a model?] Yes. Paraphrasing one definition, a model is anything thatabstracts asystem's factors in a way to helps its users quickly gaininsight into thesubject's behavior. Inaccuracy isn't a fatal blow to a model, quoting PaulWilmott's Manifesto[BW1]: • I will rememberthat I didn't make the world and thatitdoesn't satisfy my equations.• Though I will usemodels boldly to estimate value, Iwill notbe overly impressed by mathematics.• I will neversacrifice reality for elegance withoutexplaining why I have done so. Nor will I give thepeoplewho use my model false comfort about itsaccuracy. Instead, I will make explicit itsassumptions and oversights.• I understand thatmy work may have enormous effectsonsociety and the economy, many of them beyond mycomprehension. ...Navigators managed rather well with a "Flat Earth"hypothesis for sometime--no? So, we don't need over one hundred activities in ourapp. sec.model in order to provide value. [Motivation] Most of you know I respect Rohit a fair amount and so when Iread his post,you can imagine my thought, "In a world aware of BSIMM what isthe value of'Organic'?" with honest curiosity, not disdain. I immediatelyguessed> 'Organic' was meant to address a common complaint regarding almost everyprior model: "I'm challengedapplying <this> to smaller shops justbeginningtheir Application Security initiatives"Jim Bird has thoughtfully discussed the "one man shop" problemextensively> in his blog [JB1]. Rohit's own explanation mentions "no top down support" asan indication of model applicability. [Accuracy] 'Organic' ignores a lot of key components that even smallershops alreadyhave in place or care about improving. Three essential onesinclude 1)measurement and iterative approach [JB2], 2) security policy[PC1], and 3)security toolkits/frameworks [JB2][FM1]. While Rohit's postindicates> explicitly that things have been omitted, he focuses on having left outarchitecture and related activities. To me, even if 'Organic' is designed to focus only on development activities, ignoring a potential need for compliance toregulatory/security> policy, leveraging toolkits to make developers' jobs easier, or failing toset up a measure-and-iterate loop are dire mistakes. I canpoint to smallorganizations that have taken very different tacks and don'tfit the model.Some start with training. Others lean on SCR tools or securitytoolkits> before ever institutionalizing pen-testing.Perhaps it's inaccurate. Maybe it doesn't meet our industryneed foraddressing "one man shop". So is it good-for-nothing? No. It'suseful.>[Value] An immediate value that jumped out at me was the "Climb theWall" element[BS1]. I can again pull from experience organizations thatdon't fit themodel but the single most salient 'take home' from 'Organic',to me, is:You're going to needa champion, and until you "climbthe wall" you mayhit barriers to progress.This concept seems to resonate with almost every securityprofessional,> even if the form in which it comes can be contentious.Yet, often as vendors, internal champions, or <whatever> wedon't alwaysmake this requirement for progress explicit. We oftenunderestimate itscost, LoE, and impact. As a result things stall or slide backwards. Another key model elements that resonate with me is "Have a security champion":It's also worth pointing out that while the whole process growsorganically, at some stage it still requires a champion within the organization to get the organization to move to the next step.That championusually has application security in their job description, butnot always.BSIMM respondents handle this in consistent fashion [BS2] butRohit's wayof explaining this is perhaps less foreboding to those gettingtheir startand anointing their first victim <cough> champion. There's also tremendous value (to the 'Organic' model) inadmitting what Ithink we all know implicitly: programs may have to "show need"(through> assessment) in order to progress. It's not shocking to see penetrationtesting at the beginning of 'Organic'--" 'sploits alwayscreate splash" as Isay. [Conclusion] To me, the 'Organic' model suffers from key inaccuracies dueto omission.As such, it doesn't particularly address the principalcriticism of existingmodels. Its value stems from simplicity and an potentiallyclear way todrive its users through key tenets (my summary, not Rohit's): A) Anoint a champion B) Show need C) Educate execs D) Drive assessment earlier in lifecycle E) Bake assessment into BAU (QA) The ability to say these things succinctly to an organizationstarting itsApplication Security journey provides value. I do not,however, believe thatwe're going to see security assessment applied by businessunit/product teamQA folk in a BAU scenario in the next 3-5 years, with thenotable exceptionof organizations like Microsoft. If 'Organic' was mine, I would attempt to amplify itspositives byconverting it from a SDL Model to a method accompanied by casestudies. I'ddrive it towards showing, backed by case study, how "climbingthe wall" canbe accomplished most effectively. This is a tough nut and one worth cracking. -jOHN ---- John Steven Senior Director; Advanced Technology Consulting Desk: 703.404.9293 x1204 Cell: 703.727.4034 Key fingerprint = 4772 F7F3 1019 4668 62AD 94B0 AE7FEEF4 62D5 F908twitter: @m1splacedsoul Rss: http://feeds.feedburner.com/M1splacedOnTheWeb web: http://www.cigital.com Software Confidence. Achieved. * [BW1]http://www.businessweek.com/magazine/content/09_02/b4115059823953_page_2.htm> * [JB1] http://swreflections.blogspot.com/2009/04/opensamm-shows-way.html* [JB2] http://swreflections.blogspot.com/2011/06/safer-software-through-secure.html* [PC1] https://www.pcisecuritystandards.org/security_standards/ * [FM1] http://shiro.apache.org/ * [BS1] Analogous to BSIMM SM1.3 "Educate Executives": http://bsimm.com/online/governance/sm/?s=sm1.3#sm1.3 * [BS2] http://www.informit.com/articles/article.aspx?p=1434903 On Jul 19, 2011, at 11:24 AM, Paco Hope wrote:Jim, You're spot on. BSIMM is not a lifecycle for any company.Heck, it's noteven a set of recommendations. It's simply a way to measurewhat a firmdoes. It's a model formulated from observations about how somefirms'> implement software security in their lifecycles. You'll never catch uscalling the BSIMM a lifecycle.As for not translating into the SMB market, I don'tunderstand that.Unlike, say prescriptive standards which say "thou shalt do X"regardless ofhow big you are, the BSIMM measures maturity of what a firmactually does.There is no reason an SMB could not measure the maturity oftheir effortusing the BSIMM.Maturity is not a function of size. A team of 10 developersmight scorehigher on various criteria than a multi-national bank that hasa whole teamof people dedicated to app sec. Maturity is a function of thedepth to whichone takes a certain activity and their capability within thatactivity.> >This isn't Pac-Man, either. The goal is not to get thehighest score andan extra man. :) The goal is to put the right level of effortinto the rightplaces. A firm can't do that until they know how much effort they're spending on different activities. The BSIMM will illuminatethe level ofeffort. It allows a firm to decide to rebalance and spread thebudget/people> around across the activities that make sense. Whether that's a team of 10developers or a team of 1000 developers, the principle is thesame. Theexecution varies.Here's another analogy. You can have a GPS and know your exactcoordinates, to within 3 meters, but not know how to get tothe airport bycar. The BSIMM will tell you your coordinates at the presenttime. It doesnot tell you the best way to the airport. It can tell you thecrow-flydistance to the airport, but it can't tell you that theairport is where youwant to be.Paco Paco, By your same logic I would not consider BSIMM a lifecycleeither. It'sa thermometer to measure an SDLC against what some some ofthe largestcompanies are doing. As others have noted, BSIMM doesnot translatewell into the SMB market where most software is written.Don't get mewrong, BSIMM is very interesting data and is useful. But a comprehensive secure software lifecycle for every company itis not.- Jim Manico-- Rohit Sethi SD Elements http://www.sdelements.com twitter: rksethi
-- Rohit Sethi SD Elements http://www.sdelements.com twitter: rksethi
_______________________________________________ Secure Coding mailing list (SC-L) SC-L () securecoding org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates _______________________________________________
Current thread:
- Re: The Organic Secure SDLC, (continued)
- Re: The Organic Secure SDLC Anurag Agarwal (Jul 18)
- Re: The Organic Secure SDLC Gary McGraw (Jul 19)
- Re: The Organic Secure SDLC Anurag Agarwal (Jul 19)
- Re: The Organic Secure SDLC Gary McGraw (Jul 19)
- Re: The Organic Secure SDLC Rohit Sethi (Jul 19)
- Re: The Organic Secure SDLC Paco Hope (Jul 19)
- Re: The Organic Secure SDLC James Manico (Jul 19)
- Re: The Organic Secure SDLC Paco Hope (Jul 19)
- The Organic Secure SDLC John Steven (Jul 20)
- Re: The Organic Secure SDLC Rohit Sethi (Jul 20)
- Message not available
- Re: The Organic Secure SDLC Rohit Sethi (Aug 11)
- Re: The Organic Secure SDLC Gary McGraw (Jul 19)
- Re: The Organic Secure SDLC Anurag Agarwal (Jul 18)
- Re: The Organic Secure SDLC Rohit Sethi (Jul 19)
- Message not available
- Re: The Organic Secure SDLC Rohit Sethi (Jul 19)
- Re: The Organic Secure SDLC Rohit Sethi (Jul 19)