Re: Verify Your Security Provider -- The truth behind manual testing.

["Adriel T. Desautels" <ad_lists () netragard com>]

Jeff,
	I wouldn't sell our research to iDefense either, kudos to them who  
don't! But why
wouldn't a company that offers penetration testing services offer up  
any research that
it did in the form of advisories?  What is the point of doing that  
research if you never
use it to help vendors help their customers fix risks?


On Jul 17, 2009, at 8:21 PM, Jeffrey Walton wrote:

> > its just vetting of automated scanner
> > results or testing based on the results
> Hopefully the testing firm get's the merge correct during reporting by
> replacing all instances of the previous company/engagement with the
> current company/engagement.
>
> > There are some businesses that don’t actually care about
> > their security posture; they just care about passing the test
> > so that they can put a check in their compliancy box.
> It kind of reminds me of Arthur Anderson/Enron all over again.
>
> > Search vulnerability databases like milw0rm, securityfocus,
> > sirtfr, secunia, packetstormsecurity, etc. for the vendor’s name
> > to see if they have research capabilities
> Not necessarily the case. I am aware of at least one company which has
> not published an academic paper (though urged to do so), has not
> disclosed on a list, nor sold the vulnerabilities to an intelligence
> agency or broker such as iDefense.
>
> Jeff
>
> On 7/16/09, Adriel T. Desautels <ad_lists () netragard com> wrote:
> > A recent blog entry that we thought "some" of you pen-testers might  
> > find
> > interesting.  Feel free to leave comments on the blog:
> >
> > Direct URL:
> > http://snosoft.blogspot.com/2009/07/truth-behind-manual-testing.html
> >
> >
> > Verify Your Security Provider -- The truth behind manual testing.
> > Something that I’ve been preaching for a while is that automated
> > vulnerability scanners do not produce quality results and as such  
> > shouldn’t
> > be relied on for penetration tests or vulnerability assessments.  
> > I’ve been
> > telling people that they should look for a security company that  
> > offers
> > manual testing, not just automated scans. The price points for  
> > quality work
> > will be significantly higher, but in the end the value is much  
> > greater.
> > After all the cost in damages of a single successful compromise is  
> > far
> > greater than the cost of the best possible security services.
> >
> > I’ve noticed that there are a bunch of vendors who claim to be  
> > performing
> > manual testing. But when I dig into their methodologies their  
> > manual testing
> > isn’t real manual testing at all, its just vetting of automated  
> > scanner
> > results or testing based on the results. In other words they test  
> > on what
> > the automated scanner reports and don’t do any real manual  
> > discovery. I’m
> > not saying that tools like nessus (an automated scanner) don’t have  
> > their
> > place, I’m just saying that they aren’t going to protect you from  
> > the bad
> > guys. If you want to be protected from the threat, you need to be  
> > tested at
> > a level that is a few notches higher than the threat that you are  
> > likely to
> > face in the real world.
> >
> > This is akin to how the Department of Defense tests the armor on  
> > its tanks,
> > and I’ve probably mentioned this before somewhere on the blog. But,  
> > we don’t
> > test our tanks against fire from bb guns and .22 caliber pistols.  
> > If we did
> > that they wouldn’t be very effective in war.We test the tanks  
> > against a
> > threat that is a few levels higher in intensity than what they are  
> > likely to
> > face in the real world. As a result, the tank can withstand most  
> > threats and
> > is a very effective weapon. Doing anything less isn’t going to  
> > protect you
> > when the threat tries to align with your risks; you’ll end up being  
> > an
> > expensive casualty of war.
> >
> > So why do some security companies test at this lesser level? Its  
> > simple
> > really, they are in the business of making money and care more  
> > about that
> > then they do about actually protecting their customer’s  
> > infrastructure.
> > Additionally, there is a market for that sort of low quality  
> > testing. There
> > are some businesses that don’t actually care about their security  
> > posture;
> > they just care about passing the test so that they can put a check  
> > in their
> > compliancy box. Then there are other businesses that unknowingly  
> > get taken
> > advantage by of vendors because they don’t know the difference  
> > between high
> > quality and low quality services.
> >
> > So what is the difference between high quality and low quality?  
> > From a high
> > level perspective it’s the difference between real manual research  
> > based
> > security testing or not. Once hackers have access, they can do  
> > anything to
> > your data from steal it, to install back door technology in your  
> > product's
> > source code. Its happened before, and its going to happen again.
> >
> > When a company tells you that they perform manual testing hold  
> > their feet
> > to the fire. You can do the following things to verify it:
> >
> >        • Dig into their methodology and ask them specific questions  
> > about
> > how they perform their testing. (See our white papers on how to do  
> > that).
> >        • Don’t swallow jargon and terms that sound cool and don’t  
> > mean
> > anything, use Wikipedia to look up the terms and make sure that  
> > they make
> > sense.
> >        • Ask them for the names of their security experts and then  
> > use
> > tools like Google, LinkedIn, Facebook and PIPL to do research on  
> > those
> > experts. If nothing comes up then chances are their experts aren’t  
> > experts
> > at all.
> >        • Search vulnerability databases like milw0rm,  
> > securityfocus,sirtfr,
> > secunia, packetstormsecurity, etc. for the vendor’s name to see if  
> > they have
> > research capabilities. If you don’t get anything in return then  
> > chances are
> > that they don’t have research capabilities. If that’s the case then  
> > how do
> > you expect them to perform quality manual testing? Chances are that  
> > they
> > won’t be able to.
> >
> > Remember you’re putting the integrity of your business and its  
> > respective
> > name into their hands.
> >
> >        Adriel T. Desautels
> >        ad_lists () netragard com
> >        --------------------------------------
> >
> >        Subscribe to our blog
> >        http://snosoft.blogspot.com
> >
> > [SNIP]
>
> ------------------------------------------------------------------------
> This list is sponsored by: Information Assurance Certification  
> Review Board
>
> Prove to peers and potential employers without a doubt that you can  
> actually do a proper penetration test. IACRB CPT and CEPT certs  
> require a full practical examination in order to become certified.
>
> http://www.iacertification.org
> ------------------------------------------------------------------------



        Adriel T. Desautels
        ad_lists () netragard com
        --------------------------------------

        Subscribe to our blog
        http://snosoft.blogspot.com


------------------------------------------------------------------------
This list is sponsored by: Information Assurance Certification Review Board

Prove to peers and potential employers without a doubt that you can actually do a proper penetration test. IACRB CPT 
and CEPT certs require a full practical examination in order to become certified.

http://www.iacertification.org
------------------------------------------------------------------------