Re: They will protect me (won't they?)

[Sat Jagat Singh <flyingdervish () yahoo com>]

I enjoy throwing out the "alternative" perspective whenever possible to stimulate a little thinking.  So, here's the 
$0.02.

Penetration testing is one security control that should be done.  I spend most of my working time doing penetration 
tests as part of project scope.  But in some cases penetration testing is not the most important control that needs to 
be applied in the many-layered defense onion.  As a simplified example, given limited resources, should we focus more 
on doing patching or doing a penetration test that tells us we're not getting patching done?  If we are actually 
following up on our patching efforts, making sure that the patches got applied properly and tracking patch status and 
vulnerabilities for all of our installed products, applications, embedded systems, etc. as well as operating systems, 
then we will be a lot more secure than if we spend all our time and resources on penetration testing.

A penetration tester will turn up a lot of things that an auditor won't and by the same token an auditor will turn up a 
lot of issues that a penetration tester won't.  Penetration testing may reveal that there are no vulnerabilities to be 
found on a system because the admin just happened to patch it the day before the pentest project began while the 
auditor reveals that patching methodology is in an overall lousy state applied inconsistently with no testing of 
patches before hand and without proper verification of correct application.  Obviously I am simplifying again because 
there are lot of configuration related issues and undisclosed coding bugs that a skilled penetration tester may find 
but which will not be addressed with a patch.  The auditor here is not concerned with whether or not a given host is 
vulnerable on a given day, but whether a pentest was performed recently enough by a qualified tester and that the noted 
vulnerabilities were addressed
 in a proper and timely manner.

So, do vendors need to do pentests to satisfy an ethical obligation to their customers?  They should, but that probably 
isn't the security control that will give them the biggest bang for the buck.  What about following secure coding 
practices?  Do they even bother training developers on secure coding practices?  How do they maintain development vs. 
testing vs. production code libraries?  Do they have external code reviews done?  I would suggest that pentests are a 
lot more important for vendors that provide outsourcing or hosted services, because now they're playing with the 
customer's data.

Short-sightedness aside, businesses will apparently not exercise reasonable caution over information security until 1) 
$ losses leave them no choice or 2) some government regulation requires it.  I hold out no hope for number 1 to have 
much effect or we would have had a completely revamped electronic payments system long ago.  It seems like the losses 
there should be prompting some change, but instead it's considered a cost of doing business.  Performing due diligence 
in vendor selection has been a requirement for financial institutions for several years but examiners have only been 
really pushing this issue hard in the last year or so.  Exercising due diligence in vendor and product selection is 
simply not a requirement of NERC CIPs, the chemical industry CFATS or most other CIP regulations.  I'm not sure if it 
could be a part of NRC regs as I don't work in that realm, but I would doubt it.

Face it, Adriel, there will be no change of behavior by the vendors on applying penetration testing or any other 
security control until their customers demand it and the customers won't demand it unless they are mandated to by some 
federal regulation.  Even with regulatory oversight, the most that financial institutions typically do is to request a 
copy of a SAS 70 audit, which has really nothing to do with security at all.  It seems that neither the examiners nor 
the compliance officers have bothered to check the truth behind that.  They just do it because that's what other 
organizations have found they can do to put a check in the box on their security procedure.

Keep your money in bullion, dig a well and get off the grid.  Otherwise I'm afraid we're doomed.

Peace


--- On Wed, 2/11/09, Adriel T. Desautels <ad_lists () netragard com> wrote:

> From: Adriel T. Desautels <ad_lists () netragard com>
> Subject: Re: They will protect me (won't they?)
> To: "Jamie Riden" <jamie.riden () gmail com>
> Cc: "pen-test list" <pen-test () securityfocus com>
> Date: Wednesday, February 11, 2009, 4:13 AM
> Woha...
>
> First, it sounds like there is a definition problem.  Why
> are people always so unclear about definitions in this
> industry?
>
> So lets start with two basic definitions:
>
> Vulnerability Assessment:  An assessment of a target for
> the purposes of identifying weaknesses or risks in the
> target without ever attempting to penetrate or exploit those
> weaknesses. (white-box or black-box)
>
> Penetration Test: An assessment of a target for the
> purposes of identifying weaknesses or risks in the target
> and includes attempted penetration or exploitation of those
> weaknesses. (white-box or black-box)
>
> When you start talking about white-box or black-box testing
> those are methods for augmenting a penetration test or a
> vulnerability assessment and those services can be either
> black or white.  There are potentially endless ways to
> augment the testing.
>
> With respect to your comment, an auditor will never, ever
> be able to produce the same results as a penetration tester.
> If the auditor does then he's doing penetration testing.
>
>       
> On Feb 11, 2009, at 2:36 AM, Jamie Riden wrote:
>
> > Hi Adriel,
> >
> > Marcus Ranum for one disagrees  -
> http://www.ranum.com/security/computer_security/editorials/point-counterpoint/pentesting.html
> > - so I think it's a little bit misleading to say
> that all seasoned
> > security professionals think pen-test is necessary. I
> don't agree with
> > Marcus by the way.
> >
> > Fresh perspective is good, but it's also possible
> to get a fresh
> > perspective by getting an external auditor - ie. a
> white-box test -
> > rather than pen-test (black box).
> >
> > I'm obviously going to agree with your main point
> that everyone needs
> > to secure their infrastructures!
> >
> > cheers,
> > Jamie
> >
> > 2009/2/11 Adriel T. Desautels
> <ad_lists () netragard com>:
> > > Jamie,
> > >       I understand your perspective but its not
> the perspective of any well
> > > seasoned security professional.  The fact of the
> matter is that that
> > > external teams will always identify risks and
> provide new perspective that
> > > you would not get from your internal team.
> Internal teams get stale.
> > > There's a lot more to what I'm saying than
> what I've just written, but if
> > > you read between the lines I hope you understand
> where I'm coming from.
> > --Jamie Riden / jamesr () europe com /
> jamie () honeynet org uk
> > http://www.ukhoneynet.org/members/jamie/
>
>
>
>       Adriel T. Desautels
>       ad_lists () netragard com
>         --------------------------------------
>
>       Subscribe to our blog
>         http://snosoft.blogspot.com