Bugs and flaws

[Greg.Beeley at LightSys.org (Greg Beeley)]

Wietse Venema wrote:
> My experience is otherwise. Without detailed documentation I can
> usually see where in the life cycle the mistake was made: analysis
> (e.g., solving the wrong problem), design (e.g., using an inappropriate
> solution) or coding.

I tend to agree - for *many* design related problems.  But I think it
is only true for design flaws that are violations of well-recognized
approaches to things (for instance, putting too much trust in a source
IP address for authentication, or blatant misuse of cryptography), or
when the problem being "solved" by the software is self-evident enough
that the auditor essentially repeats much of the software engineering
process, albeit (possibly) very informally, just by auditing the code.

Other design related defects are hard to find if you don't have a
well-defined problem - the old "validation" vs "verification" issue.
When the problem being solved by the software is an uncommon one, or
unique to the software, it is more likely that a design flaw will go
undetected by an auditor (for instance, your average code auditor
won't catch a design flaw in how retinal scanning software authenticates
a person, without having studied how it is supposed to work in the
first place).

- Greg

03-Feb-2006