Re: Evaluating pentesters

[Pete Herzog <lists () isecom org>]

Hi,

> Or maybe have a yearly competition from the guys at ISECOM and publish
> the results.

Thanks for the suggestion! However, in reality most pen-test companies 
won't subject themselves to such a bake-off because they might not win 
and they certainly don't want to be published as a loser. We have 
worked on some ways to show who's good without showing who's bad and 
while I think we can do regular bake-offs, I think a good security 
company is found in their accountability rather than any mythical or 
even valid but poorly quantifiable criteria. Remember, any Tester can 
have an off day and any Analyst can miscommunicate results. 
Furthermore, in a thorough security test, which is about evaluating 
balanced protection as action items for remediation, there is less to 
say about who's better as all should be fairly equal at such a task 
(the discrepancy is often a result of being able to afford commercial 
tools to do certain things faster or more in depth but not necessarily 
more thorough or better). So what it comes down to in the end is 
Accountability: who's going to sign off and stand by what they have 
determined and who's going to cover their asses with circular talk and 
bull about no stopping persistent hackers....

So if you want to make sure you get the security test you want, ask 
for them to do a STAR (Security Test Audit Report). It provides 
accountability without forcing companies to compete. We'll be 
releasing the newest version soon but the original version is still 
available here: www.isecom.org/ravs

The STAR will have them sign off on their findings as well as clearly 
state what was not tested. All this is well explained in OSSTMM 3 that 
we've wrapped up and are preparing now for public release.

Sincerely,
-pete.

Managing Director, ISECOM

------------------------------------------------------------------------
This list is sponsored by: Information Assurance Certification Review Board

Prove to peers and potential employers without a doubt that you can actually do a proper penetration test. IACRB CPT and CEPT certs require a full practical examination in order to become certified. 

http://www.iacertification.org
------------------------------------------------------------------------