RE: Penetration testing via shrinkware

[Gary Crumrine <gcrum () us-state gov>]

Yes exactly Ryan.  Too many times we see this as a battleground where 
we capture the flag by gaining root.  When what a client is really 
paying for is for us to identify vulnerabilities and perhaps suggest 
ways to close the gaps.  When QAing a given "system", you are dealing 
with multiple servers, workstations etc.  In this case, using 
automated tools make sense since they allow you to view and assess 
more machines in a limited amount of time.  At least they give you 
the obvious configuration errors.  The most important outcome of a 
certification/penetration test, is that you raise the awareness of 
the client and their staff.  So if you certify a site today, it only 
means that they are good on that day.  Chances are that tomorrow or 
next week, a modification to the system will occur that could place 
the enterprise at risk again.  But if you make the administrators 
more aware of the possible outcomes for taking certain actions, they 
may at least think about it for more than a nano second.


-----Original Message-----
From:   Ryan Russell [SMTP:ryanr () sybase com]
Sent:   Thursday, September 03, 1998 1:12 PM
To:     Stout, Bill
Cc:     Firewall-wizards
Subject:        Re: Penetration testing via shrinkware




> What are the opinions on the thoroughness of shrinkwrap software
> penetration testing?  Is today's shrinkware more capable for 
penetration
> testing (a single machine) than a human?

Depends on the human.  Even when compared to a really good human,
the software will often find a hole the human didn't think to check 
for,
didn't
know about, or didn't care about.

I think they're actually good for different things... You want a 
person
driving a penetration
test, who can do all the things you've mentioned, and use their head, 
and
correlate
information a program couldn't begin to.  From the case studies I've 
read,
it seems
human penetration tests tend to be "I got root, game over."  In other
words, the point
is to prove there is at least one way in, not neccessarily to 
enumerate ALL
the ways
in.

You want to use the software to do mass checking of hosts and 
problems.
Unlike
the human who tend to want to find the one big hack, the software is 
happy
to report
small things, potential problems, and things that aren't "broken" 
exactly,
but
just don't follow policy.  An example would be ISS's ability to check 
that
NT hosts
enforce the minimum password length that you want users to use.  I 
see the
software
as being more useful that a person when trying to close down as many 
holes
as
possible on many hosts.

Another point of discussion about the software is that it tends to 
*find*
holes,
and not neccessarily *exploit* them.  The software vendors do this
intentionally
to prevent liability to some degree.  (I think the next big Internet 
worm
will be
a customized SATAN or SAINT that performs the attacks it checks for, 
and
then installs itself to go after the next host.)

Of course, what you really want is a really good human armed with the
software.


                              Ryan