Functional Correctness

[chandra at list.org (Pravir Chandra)]

Well, this topic gets muddy pretty quickly since I agree with many of
the comments made on this thread. We have to be careful with hype and
claims made by new models (BSIMM and OpenSAMM in particular) since
depending on how the 'rest of the world' sees them speaks directly to
our credibility as industry experts.

I've tried hard when presenting OpenSAMM to fully claim that the model
is chocked full of value judgements about what organizations SHOULD be
doing to make a justified argument (qualitatively) that the software
they produce has a degree of assurance built-in. Is it a guarantee?
No. Is it still valuable? Absolutely. Before, we had no ability to
make an apples-to-apples comparison between two organizations, and the
model helps that. We also didn't know how to quantify iterative
improvement very well or explain the breadth of the software security
problem to people either, and OpenSAMM helps that too. I disagree with
the remark that maturity models are only useful to companies starting
with nothing, because I've seen firsthand how OpenSAMM has helped
people (already doing a lot for assurance) think through aspects of
the software security problem that fell outside their tunnel-vision.

Now, on to the sticky topic of value judgements. Based on how I've
seen the BSIMM presented, one might think that at face value, it is
somehow more free of author/contributor value judgements than OpenSAMM
or other secure SDLC models (I've read several articles referring to
these as 'alchemy'). This is simply not true. I, for one, agree with
Brad that claims of a scientific nature need to be extremely carefully
qualified. At the end of the day, we don't yet know enough about
practical methods for improving software security that have much
justification beyond what experts think amounts to a 'good thing'
(excepting formal methods, of course, but I did say practical :). This
is the case for both BSIMM and OpenSAMM.

I welcome comments/questions/flames.

p.





On 8/22/09, Cassidy, Colin (GE Infra, Energy) <colin.cassidy at ge.com> wrote:
> Brad Andrews Writes:
>
> > After all, we can just "implement this maturity model and eliminate
> > all our security problems, at least in the application,
> > right?"  That
> > is likely to end up resulting in even more resistance in the future
> > when management questions why they need to keep spending more for
> > software security, a secure architecture, etc.  Don't people learn
> > what they need to know at some point?
>
> I don't thinks that's ever been the case that you can just apply your model
> and all will be well Microsoft didn`t release their SDL and said "there all
> our software will now be secure", they're constantly evolving their
> processes.
>
> Also some of the activities within the BSIMM are about constant improvement
> and keeping up with the latest trends, so even just following the BSIMM your
> processes are never static.
>
> > I don't think we will ever be static.  As soon as we remove the low
> > hanging fruit, the fruit higher up the tree will be the problem.
>
> Or, the fruit on another tree :) who's attacking the OS now when the apps
> are so easy to attack
>
> > This isn't to say a maturity model is useless, but I remain
> > skeptical
> > that it will live up to the "hype" (low key now, but there) it is
> > being presented with.
>
> I think that the models (both BSIMM and OSAMM) help to provide a framework
> and a direction to those that have no real security practices at all.  Or
> allow a measurement of existing process and see where their weaknesses are.
> That and the senior management like the pretty graphs even if they don't
> know what it means :D
>
> CJC


-- 
~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~ ~~~~~~~~ ~~~~~ ~~~ ~~ ~
Pravir Chandra                      chandra<at>list<dot>org
PGP:    CE60 0E10 9207 7290 06EB   5107 4032 63FC 338E 16E4
~ ~~ ~~~ ~~~~~ ~~~~~~~~ ~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~