Re: Evaluating pentesters

[Rudra Kamal Sinha Roy <rudrak () gmail com>]

Hi Tony,

Have a look at this blog post : "5 Tips on Choosing Penetration
Testing Companies" :
http://www.ivizsecurity.com/blog/penetration-testing/how-to-choose-penetration-testing-companies/
 Go through it carefully and it will answer all your queries, I hope.
The tips which are highlighted in this post are:

Tip 1: Evaluate Technology Competence of Vendors
Tip 2: Focus on the vendor’s real knowledge and not just on certifications
Tip 3 Evaluate the company’s trustworthiness and competence
Tip 4: Consider cost versus frequency maximum leverage
Tip 5: Seek penetration testers (Specialists) and not Generalists

Thanks!

Rudra Kamal Sinha Roy


On Tue, Mar 9, 2010 at 4:33 AM, Shohn Trojacek <trojacek () gmail com> wrote:
> Tony,
>
> I'd say that similar to a job interview, you could ask them to tell
> "war stories" and then measure their hesitation and response time to
> detect BS. Of course, you don't want to mistake contemplation for
> hesitation, but this is generally an effective tool in any area. For
> example, you can call up a former employer and ask if they would hire
> that person again. The lack of a response can be more telling than an
> actual response at times.
>
> So essentially, the process is filter based on sample report,
> methodology, etc. This is basically like looking at someone's resume.
> Perhaps you could ask about certs, but then that may not mean anything
> either. Then once you have screened the Nessus repackagers, interview
> them placing an emphasis on war stories or perhaps describing a
> scenario and evaluating the thought process.
>
> If you issue an RFP and such, I imagine you could just bake this into
> the process depending upon your organization's contraints and such.
>
> Of course, I've found that often people request a "penetration test"
> and really want they want is exactly what you don't want.
>
> Often they just want a Nessus scan repackaged so that they can check
> whatever box they are required to. This comes back to defining what
> you want. I like to use the terms "creating management awareness of
> the depth of issues by demonstrating pro-longed and undetected access"
> in conjunction with breadth by perhaps requiring "cross-checks and
> verification of the results of scanning tools".
>
> Your mileage may vary and each situation is unique usually.
>
>
> Shohn
>
>
> On Fri, Mar 5, 2010 at 6:01 PM, Tony Turner <tony_l_turner () yahoo com> wrote:
> > Is there some kind of "Who's Who" of penetration testing firms? Right
> > now my primary methods for evaluating potential firms for pentest
> > engagements are requesting sanitized reports from past tests and asking
> > questions about their methodology. Is there some resource online I might
> > be able to use to locate quality testers? I've been burned in the past
> > with some real bad ones.. I'm looking for
> > network/systems/application/web/wireless from a PCI focused firm. Not so
> > much interested in physical security and social engineering tests at
> > this time but these services may be useful for future engagements. Also
> > not interested in paying good money for someone else to just do a
> > Kismet/Gpsmap or Nessus scan for me and hand me the scan data. Useful
> > tools of course, but I've met a few idiots who thought that was what
> > penetration testing was. I am in the SE United States.
> >
> > --
> >
> > Tony L Turner
> > CISSP, CISA, GPEN, GCIA, GSEC, VCP, ITIL-F
> >
> > ------------------------------------------------------------------------
> > This list is sponsored by: Information Assurance Certification Review Board
> >
> > Prove to peers and potential employers without a doubt that you can actually do a proper penetration test. IACRB 
> > CPT and CEPT certs require a full practical examination in order to become certified.
> >
> > http://www.iacertification.org
> > ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------
> This list is sponsored by: Information Assurance Certification Review Board
>
> Prove to peers and potential employers without a doubt that you can actually do a proper penetration test. IACRB CPT 
> and CEPT certs require a full practical examination in order to become certified.
>
> http://www.iacertification.org
> ------------------------------------------------------------------------

------------------------------------------------------------------------
This list is sponsored by: Information Assurance Certification Review Board

Prove to peers and potential employers without a doubt that you can actually do a proper penetration test. IACRB CPT 
and CEPT certs require a full practical examination in order to become certified.

http://www.iacertification.org
------------------------------------------------------------------------