Re: Penetration testing via shrinkware

["Joseph S. D. Yao" <jsdy () cospo osis gov>]

> I have never seen any data that suggest that formally-evaluated systems are
> (much, if any) higher quality than non-evaluated systems.  ...

Interestingly, the largest base of peer-reviewed software around
appears to be much more stable than that most commercial systems of any
kind.  I'm referring, of course, to Open Source software - GNU, *BSD,
Linux, et al.  It also affords much quicker upgrade turn-around time.

Of course, "stable" has more than one meaning ... and that same
software is far from stable, if you want a product that doesn't change
for a long period of time [and thus has no new bugs introduced].

Right now, there's an attempt to do a Security Audit on Linux and the
software that runs on it.  It's interesting - perhaps someone should
document it.  Despite the "peer review", there had previously been no
formal effort to remove vulnerabilities.  And they're finding potential
vulnerabilities - or they think they are - right and left.  This from
the system I had just described as more stable than most.

But the people doing this are just volunteers with no special training.
In fact, from what I can tell, their abilities range all over the map.
It's possible that "cookbook vulnerabilities" that some of them are
finding really aren't.  It's also possible that they're missing some.

What does this mish-mash of observations tell us?  Nothing new.  The
art of software development is still in its childhood.  Programmers
abound, but software engineers - or programmers who use a software
engineering approach - are few and far between.  There is no method
that is "foolproof"; and if one existed, it probably wouldn't be
"damfoolproof".  Peer review is wonderful, but first you need (a)
something against which to review (specifications?  design?), (b)
perhaps a methodology, (c) certainly a methodological approach, and (d)
probably some talent.

And formal proofs are behind even that curve.

;-)

--
Joe Yao                         jsdy () cospo osis gov - Joseph S. D. Yao
COSPO Computer Support                                          EMT-A/B
-----------------------------------------------------------------------
        PLEASE ... send or Cc: all "COSPO Computer Support" mail to
                        sys-adm () cospo osis gov
-----------------------------------------------------------------------
This message is not an official statement of COSPO policies.