Re: Penetration testing via shrinkware

[Crispin Cowan <crispin () cse ogi edu>]

John McDermott wrote:

> > A scanner, on the other hand, is simply not possible to verify.  No matter
> > what vulnerabilities the scanner checks for, there will always be the
> > potential for a new mis-configuration, bug, or other vulnerability in some
> > product that the scanner should check for, but does not.  The set of
> things
> > that a scanner should check for is infinite, so the scanner can never be
> > complete.
>
> By the same token, how can firewall testing be accomplished?  Let us assume
> bug B.  If there is no scanner for bug B because it is unknown until time
> T, then how can a firewall be certified at time <T that it protects itself
> and an internal network from bug B?  That is, testing goes hand-in-hand
> with firewall certification, as I see it.

Verification need not be confined to testing.  You could also do
FORMAL verfication, which involves inspecting the source code, and proving
mathematically that there are no bugs at all.  Let me be perfectly clear:  I
do NOT regard this as a practical approach, I am just observing that it is a
theoretical possibility.  Very few organizations have the resources to persue
A1 certification for a product of any complexity.  But it is theoretically
possible to prove that a firewall is bug-free.  It is not theoretically
possible to show that a scanner can detect all bugs.

TBQF observes that I have a mis-conception about scanners, asserting that a
scanner's stated purpose is to scan for a finite list of bugs, not all
possible bugs.  Fair enough, if that is what is meant by "verifying" a
scanner, then I agree that it is theoretically possible to achieve
verification that a scanner can reliably detect a finite list of bugs.  It
just makes the idea of verifying a scanner a whole lot less interesting.

Crispin