Re: BSIMM-V Article in Application Development Times

[Stephen de Vries <stephen () continuumsecurity net>]

Hi Sammy, Antti,

On 20 Dec 2013, at 17:29, Sammy Migues <SMigues () cigital com> wrote:

> Also, in nearly all cases, it would be very hard to characterize an entire firm or even an entire business unit in 
> larger firms as "Agile" or not. Many larger firms use "Agile" for only a small percentage of projects 


Leaving the definition of agile aside for the moment, doesn’t the fact that the BSIMM measures organisation wide 
activities but not individual dev teams mean that we could be drawing inaccurate conclusions from the data?  E.g.  if 
an organisation says it is doing Arch reviews, code reviews and sec testing, it doesn’t necessarily mean that every 
team is doing all of those activities, so it may give the BSIMM reader a false impression of the use of those 
activities in the real world.

In addition to knowing which activities are practiced organisation wide, it would also be valuable to know which 
activities work well on a per-team or per-project basis.

On 17 Dec 2013, at 22:01, Antti Vähä-Sipilä <avs () iki fi> wrote:
> Moreover, I think this sort of split would be largely arbitrary. Especially for large companies, it's often not 
> straightforward to classify them as agile or non-agile. Many companies also have mixed-mode dev shops with waterfall 
> product management bolted on top of an agile dev team, or an agile dev team throwing code over the wall to a 
> traditional ops team, or a mix of agile and non-agile teams working side by side. 

Agree that the split between agile and not-agile would be arbitrary at the organisation wide level.  But deciding on an 
arbitrary line, or better yet an arbitrary scale of agility on a per-project level shouldn’t be too difficult.  If we 
need to start somewhere, then I think borrowing from devops couldn’t hurt, where they measure agility by:
- frequency of code deployments
- lead time from code deploy to running in production

> In addition, I don't think you can measure agility through purely measuring cadence. The point of being agile is to 
> be able to respond to change, and not all companies _need_ to be reinventing their product daily like a budding 
> startup with an existential crisis. Although continuous integration would probably help the majority of companies, on 
> the product management (i.e., backlog management) side, it depends on your customers and industry whether more is 
> indeed better.

With the BSIMM’s objective of just describing activities it wouldn’t be necessary to promote agile or agile security 
practices.  But it would be interesting to know that if an organisation happens to have chosen agile or continuous 
delivery as their software dev methodology, then how are they integrating security into that process?  The burning 
questions I have regarding agile and continuous delivery and security are:
- What mixture of the BSIMM activities work well in a continuous delivery style environment?
- As you move from less-agile to more-agile, which activities tend to fall away and which are more emphasised?
- How are the security specialist and time heavy activities like attack models, sec arch review and pentesting 
performed when new code is pushed to production daily?
 
The BSIMM seems to be the only place where this type of data exists or could be captured- so would be nice to be able 
to extract this data from it; or include these types of questions in future versions.  The devops survey(*) is another 
potential, but as yet they don’t capture security specific activities.


* http://itrevolution.com/the-science-behind-the-2013-puppet-labs-devops-survey-of-practice/


regards,
Stephen

_______________________________________________
Secure Coding mailing list (SC-L) SC-L () securecoding org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates
_______________________________________________