Re: Pen Test vs. Health Check

[Ivan Arce <ivan.arce () coresecurity com>]

Robert E. Lee wrote:

> > A Pen Test is only as good as the testers and is only a snapshot.

Ehm I belive that is the common understanding of the practice as it is
right now, but not necesarily the only way to look at what a pen-test
can be used for. I will elaborate on this further in the following paragraphs.

> I can not argue that the test is only as good as the tester/analyst team,

This is so just because the practice *as it is right now* is based on
the individual skills of the testers.

> but the output if prepared and analyzed properly the results can far outlast
> the time value of "snapshots" I've seen delivered.  A snapshot might uncover
> a set of patches the customer didn't have installed, but might miss the fact
> that there may be a security concern with the patch management policy of the
> tested organization.  Obviously you can not talk about a new vulnerability

Granted, but that is only the case if the results of the pentest and the
report elaborated and presented thereafter is written without leveraging the
expertise and experience of the testers. A good pen-tester will be able
to extrapolate the vulnerabilities and misconfigurations found in a
given pen-test and identify the root of those problems and not only the fact
that vulnerability XYZ is present and exploitable because patch P was
not installed on set W of boxes.
An experienced 'attacker' will understand this and other problems as the 
symptoms of bigger and more serious issues than need to be addressed and
will report them as general conclusion and suggest solutions.
Specially if the pen-test is repeated on a periodic basis and the 
vulnerabilities exploited in each one tend to be of the same nature.

> When I talk about a pen-test it is only to act as a proof of concept for
> what might be possible if a real attack were to occur.  My goal in that case
> is maximum damage... find as many trophy's (client list, ssn/financial db,
> root access, etc) as possible.  This type of test can serve as a wake up
> call, but doesn't provide any other lasting value.  Restated a pen-test's
> goal is to find the weakest link and the maximum exposure possible.

Ok, so here you yourself outline how pen-test can become a more useful 
practice if thought of as part of a bigger security process. If pen-tests
are executed on a periodic basis and as part of an iterative process that
includes:
 1. Do a penetration test, ie. find the weakest(s) link(s)
 2. Fix the problem (close the weakest links or paths into your valuable
    assets
 3. Audit the countermeasures deployed (verify that things are working
    properly and that your patches, ACLs and firewalll rules, IDS systems,
    antivirus, etc are monitorable and provide enough information to detect
    their own failures

 4. Goto 1


In this manners although you wont *ever* achieve 100% security you will be
sure that you a have a working security process  that will constantly
improves your security posture in a timely manner and whioch is inline with
the day to day status quo with regards to your organization, new
vulnerabilities, attack trends and the way your attacker (and you need to
define this according to your threat model) takes advantage of your 
particular weaknesses.

As much and as quick as you interate in this process the better
your security posture will be. Looking at pen-testing in this way and as
part of a bigger process, the "snapshot" view of a pen-test is still
valid for a single instance but no longer holds when you take a set
of consecutive tests over a period of time.

The fact that pen-tests are expensive and resource intensive have prevented
security practitioners from its adoption as a common and regular practice,
but that is a problem not intrinsic to the underlying philosophy of
attacking yourself in order to improve your defenses, its as shortcoming of
the current state of the practice and the technologies used to deliver it.

-ivan

---
To strive, to seek, to find, and not to yield.
- Alfred, Lord Tennyson Ulysses,1842

Ivan Arce
CTO
CORE SECURITY TECHNOLOGIES

46 Farnsworth Street
Boston, MA 02210
Ph: 617-399-6980
Fax: 617-399-6987
ivan.arce () coresecurity com
www.coresecurity.com

PGP Fingerprint: C7A8 ED85 8D7B 9ADC 6836  B25D 207B E78E 2AD1 F65A



---------------------------------------------------------------------------
----------------------------------------------------------------------------