Re: Penetration testing via shrinkware

[Ted Doty <ted () iss net>]

At 12:44 AM 9/18/98 -0700, Crispin Cowan wrote:
> tqbf () pobox com wrote:
> > Scanners are probably much easier to certify than firewalls (which
> > probably can't be meaningfully certified at all).
>
> I beg to differ.  A firewall can at least theoretically be verified:  if
it is
> formally proven to enforce a policy of (say) allowing through traffic on
ports X
> and Y, and no others, then the firewall is verified.  A scanner, on the other
> hand, can never be verified, because the potential list of vulnerabilities
that
> it could reasonably be expected to check for is infinite.  Scanners can
never be
> complete, because the space of possible mis-configurations and buggy software
> knows no bounds.

The problem is that creating these models is not trivial (or inexpensive).
Many times the model will not even work without a simplifying assumption,
but the assumption does not completely model the Real World.  The Orange
Book required systems certified for A level (the highest level) have a
formal security model that the certification team would use to prove or
disprove, so there is a body of experience directly applicable to this
problem. [anyone out there survive an A-level Orange Book evaluation?]

I remember many learned dissertations in the 1980s on Why Ethernet Will
Never Run Faster Than 2 Mbps.  These typically made assumptions like:
Ethernet is p-persistent (or non p-persistent, can't remember which), which
warped the collision behavior.  Same thing happened with models of TCP
state behavior, which proved we would never run faster than 1000 packets
per second.

Models are great if you're looking for an initial ballpark estimate (prior
to subsequent Real World analysis), or if Real World analysis is too
expensive or inconvenient to do (say, evacuation models for Nuclear power
plants).  Don't think firewall testing falls easily into either of these
categories.

You are correct that scanners can never be complete, but this strikes me as
true for just about all software products (including firewalls).

- Ted

-----------------------------------------------------------------------
Ted Doty, Internet Security Systems          | Phone: +1 678 443-6000
6600 Peachtree Dunwoody Road, 300 Embassy Row | Fax:   +1 678 443-6479
Atlanta, GA 30328  USA                       | Web: http://www.iss.net
-----------------------------------------------------------------------
PGP key fingerprint: 362A EAC7 9E08 1689  FD0F E625 D525 E1BE