Marcus Ranum downplays importance of Pen Test Tools like Metasploit - opinions?

[paul at pauldotcom.com (Paul Asadoorian)]

Comments inline...

> So I was listening to the Risky Business Podcast this AM (#85) on my
> commute in (right after finishing part II of pauldotcom) and they had
> Tenable Network Security's CSO Marcus Ranum on. 

I won't comment on the order in which you listened to the podcasts ;)

> Marcus stated that he
> felt tools such as Core and Metasploit had no usefulness in pen test. 

There are just so many things wrong with this statement.  The exploit
frameworks (Metasploit, CANVAS, and Core IMPACT) provide the penetration
tester with the following:

- Reliable exploits
- Re-usable payloads
- Payload features (Meterpreter, MOSDEF, and the Core Agent have many,
many useful features, too many to list all of them here, but encryption
between compromise host and framework, SAM database dumping, and in-line
shells are just a few)
- Reporting (not all have reporting)
- Exportable payloads, so in a pen test you can compromise a machine
however you like (USB thumb drive, weak passwords, web application) and
deploy a custom payload
- Provides a framework for exploit development

If I were procuring a penetration test, I would make certain the person
I hire is comfortable using at least one of the frameworks.

> He
> emphasised that a design review and vulnerability scanning should be
> enough.

Design review, and even forms of vulnerability scanning, can miss so
much.  How do you know the patch you just rolled out was successfully
installed on every host in your environment?  How do you really know the
configuration on your routers that implements security feature X is
working?  Is that data really encrypted on that protocol you chose to
use?  You can point at a document and say "Yes, we are secure!", or you
can actually test it and find out for real.  Imagine if TSA adopted this
model, they could just come out and say, "Yup, we installed x-ray
scanners at every airport and follow these procedures, so we're secure".
 If they never really test it, how do they continue to improve?

> While I may have misunderstood his statements and I do agree
> design/config reviews and vulnerability scanning needs to be the first
> and second step of any regular review, pen test, etc... 

Don't get me wrong, vulnerability scanning and config/design reviews are
important, you should do them.  In my previous jobs as a network
security engineer we spent a signifigant amount of time designing our
security architecure, and evolving it (sometimes with the help of
external consultants).  So, yes, this is important, but you have to "put
the rubber to the road" and test it at some point.  If you have an area
where you know your security is lacking, have gotten approval to fix it,
and are in the process of implementing a fix, then there is no need to
pen test it :)

> Nessus is not going to
> tell me if my blackberry user is connecting to free wifi and is
> vulnerable to Karma, etc..

True, thats is a good point, there are specific technologies where its
important to understand the risk.  Oh, and did I mention that the goal
of a pen test is to help evaluate and understand risk and business
impact?  That a test only begins once you exploit a machine? :)

Cheers,
Paul
-- 
Paul Asadoorian
PaulDotCom Enterprises
Web: http://pauldotcom.com
Phone: 401.829.9552

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 257 bytes
Desc: OpenPGP digital signature
Url : http://mail.pauldotcom.com/pipermail/pauldotcom/attachments/20081030/dbf0ddee/attachment.pgp