Re: Agile (Scrum) best security practices and experiences?

[Antti Vähä-Sipilä <avs () iki fi>]

Dave said:
> I then did a reprise/updated version at OWASP AppSec US in NY in 2008.
> The slides and a video of the presentation are available here:
> http://www.owasp.org/index.php/OWASP_NYC_AppSec_2008_Conference 

I did see Dave's OWASP slides earlier, but for some reason I never found or watched the video previously, which really 
has much more information than just the slides. I should have! That's good stuff.

I agree with the security stories and 'abuser stories' (misuse cases) being very useful and how product management can 
come up with those was one of the things I also tried to discuss in the Stockholm OWASP presentation. I think that 
having a way for product managers to come up with security stories independently of security expert help is necessary 
especially in a large organisation where the requirements are being managed all the time (instead of a "sprint 0" , our 
requirements management is continuous).

About dedicated security sprints - I've never seen them done myself, but two of the agile gurus I've listened to 
advised against 'themed' sprints in general. This is because if you know you'll have a security sprint coming up (or a 
performance sprint, or any sort of other quality sprint), there's a tendency to push that kind of work forward into 
that sprint, thereby creating quality debt. Also one of the underlying ideas of Scrum is that the code should be 
shippable and 'done' from quality perspective after every sprint - this is also what Erlend had in his presentation 
about Definition of Done. Postponing this work to some upcoming sprint just seems to violate this principle. This is 
also the question I have with Microsoft's Agile SDL. I wonder if their bucket-based system creates this sort of quality 
debt. But perhaps a dedicated sprint would work for some, if it is frequent enough.

Related to the above, plus this Rohit's excellent comment:
> * Threat modeling needs to be agile and done in a matter of hours
> rather than days. The focus should generally be on important high/risk
> use cases rather than attempting to be comprehensive


We've building much of our SDL on the notion of threat modelling (we call it security threat analysis). Instead of a 
special sprint or a dedicated modelling session, what we currently advocate is doing the design/implemantation level 
threat analysis on backlog item level. Threat modelling would be interleaved as small backlog items before the 
respective implementation backlog items. This seems to be called a 'spike' in some agile literature. We feel this 
retains the agility better, avoids the quality debt from accruing, avoids wasted analysis if the requirements change, 
and generally makes the security analysis work visible in task scheduling instead of being something "on the side".

The downside is that threat modelling on backlog item level may sometimes have a focus that is too narrow. Typically it 
will be team-wide anyway (because intelligent people take into account what they anticipate they'll be working on in 
the future), but sometimes you'd need more, like a cross-team view. This may be a problem, and could be alleviated by 
an increased investment in the higher-level abuser stories.

Dave also said:
> I felt that my talks were kind of from the macro view of the problem
> (i.e., top level down), where Erlend's talk was from the micro view
> (bottom up)

I think the macro discussion is really essential here. I mean, most coding / testing level security engineering 
activities can be applied in waterfall just as in agile, and adding some specific coding or testing activity does not 
immediately break the agile work scheduling. But when it comes to risk management (threat analysis/modelling, and 
residual risk acceptance), security assessments or security committee mandated quality gates have, if done wrong, all 
the potential to destroy all the agility and lean-ness of the model.

Rohit also raised a number of other very good points. Some comments:

> * As with all other types of SDLC, education is a critical
> prerequisite to convince stakeholders of the importance of security

I would actually even go farther and say that in an agile development model, education (or at least awareness raising) 
is even more important than if you'd have a centralised security team that would guard a quality gate in a waterfall 
process. This is because a large agile project is by its nature decentralised, with some residual risk decisions taken 
to the engineer level (a Sprint Review's definition of 'done' is in a way a residual risk acceptance gate). Also, in a 
large project consisting of several teams, the sprints are often synchronised, with a number of teams doing their 
sprint planning and review at the same time. You'd need a lot of security experts at the same time to sit in the 
planning sessions.

> * Documentation, such as secure coding guidelines / checklists,
> should reside in dynamic platforms such as a wiki or web page rather
> than static documents that don’t evolve

Also threat analysis results benefit from being in a wiki. Especially when threat modelling is frequent and iterative.

In the end, the threat information and residual risk information (i.e., what wasn't fixed) would accrue on a single 
wiki page from several sources: the teams would fill that in when doing design and implementation level analysis, the 
Product Owner would add more information when thinking about abuser stories and security stories.

Cheers,

Antti


_______________________________________________
Secure Coding mailing list (SC-L) SC-L () securecoding org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates
_______________________________________________