Re: Penetration Test Assistance

[Brett Watson <brett () the-watsons org>]

On Jun 5, 2012, at 9:52 AM, Peter Kristolaitis wrote:

> As far as horror stories... yeah.   My most memorable experience was a guy (with a CISSP designation, working for a 
> company who came highly recommended) who:
>    - Spent a day trying to get his Backtrack CD to "work properly".  When I looked at it, it was just a color depth 
> issue in X that took about 45 seconds from "why is this broken?" to "hey look, I fixed it!".
>    - Completely missed the honeypot machine I set up for the test.  I had logs from the machine showing that his 
> scanning had hit the machine and had found several of the vulnerabilities, but the entire machine was absent from the 
> report.
>    - Called us complaining that a certain behavior that "he'd never seen before" was happening when he tried to nmap 
> our network.  The "certain behavior" was a firewall with some IPS functionality, along with him not knowing how to 
> read nmap output.
>    - Completely messed up the report -- three times.  His report had the wrong ports & vulnerabilities listed on the 
> wrong IPs, so according to the report, we apparently had FreeBSD boxes running IOS or MS SQL...
>    - Stopped taking our calls when we asked why the honeypot machine was completely missing from the report.
>
> In general, my experience with most "pen testers" is a severe disappointment, and isn't anything that couldn't be 
> done in-house by taking the person in your department who has the most ingrained hacker/geek personality, giving them 
> Nessus/Metasploit/nmap/etc, pizza and a big ass pot of coffee, and saying "Find stuff we don't know about. Go.".   
> There is the occasional pen tester who is absolutely phenomenal and does the job properly (i.e. the guys who actually 
> write their own shellcode, etc), but the vast majority of "pen testers" just use automated tools and call it a day.  
> Like everything else in IT, security has been "commercialized" to the point where finding really good vendors/people 
> is hard, because everyone and their mom has CEH, CISSP, and whatever other alphabet soup certifications you can 
> imagine.

I agree with a lot of what you've said, but there are absolutely good security guys (pen tester, vulnerability 
assessors, etc) that use both open source and commercial automated tools, but still do a fantastic job because they 
understand the underlying technologies and protocols.

I used to do a lot of this in the past, had lots of automated tools, and only occasionally wrote some assessment 
modules or exploit code if necessary.

But again, a person in that position has to understand technology holistically (network, systems, software, protocols, 
etc).

-b