RE: Fwd: External Pentests Obsolete?

[<richard () bluesec net>]

In this context I would refer people to OSSTMM as a methodology where
testing can be carried out from multiple scopes and security domain
boundaries (both directions) and results can be accumulated to and overall
rating which can be compared across tests. 

All too frequently I am seeing pen test reports (and audits) that are just
so narrow that the customer has very narrow vision of the extent of the risk
and ends up chasing A when really B, C and D are more important, but weren't
tested.

I guess the real question is actually... What have 'we' been selling ...
Clever pen testers or a report accuratly describing the risk position of the
customer ?

Craigs point about compliance ... In essence you are checking that you
comply to something. This will rarely be shown by a pen test. A full and
proper assessment method / process is needed and if I have it right this
will turn into something that needs to be reported on a day to day basis. 

Richard

> -----Original Message-----
> From: listbounce () securityfocus com 
> [mailto:listbounce () securityfocus com] On Behalf Of 
> cwright () bdosyd com au
> Sent: Monday, 13 August 2007 9:59 a.m.
> To: pen-test () securityfocus com
> Subject: Re: Fwd: External Pentests Obsolete?
>
> Actually, PCI-DSS requirements specify more than a Pen Test. 
> It does require the scanning of interfaces, but not in the 
> manner that is being associated with a Pen Test.
>
>
>
> The Firewall also includes " 1.1.8 Quarterly review of 
> firewall and router rule sets". This is not just an external 
> scan. It requires the validation of egress filters, an action 
> not possible through a Pen Test. The scanning process - 
> https://www.pcisecuritystandards.org/pdfs/pci_scanning_procedu
> res_v1-1.pdf is not something that could be called a Pen Test 
> anyway. It is an external facing interface vulnerability assessment.
>
>
>
> What people seem to ignore is that all merchants and 
> providers are REQUIRED and contractually obliged to meet the 
> standard. The differentiation is the standard of proof that 
> it is being met, not if it needs to be met.
>
>
>
> A pen test will not determine compliance with any of the 
> following PCI-DSS requirements:
>
> 1.1.5 Documented list of services and ports necessary for business
>
> 1.1.9 Configuration standards for routers.
>
>
>
> Even:
>
> 1.3.2 Not allowing internal addresses to pass from the 
> Internet into the DMZ
>
> Will be difficult using a Pen Test methodology. The ideal is 
> to test packet flows by validating the firewall. This 
> requires that a sniffer is setup on DMZ and internal 
> networks. Not a part of a Pen test.
>
>
>
>
>
> I could point again to research that I have led in the past. 
> An effective review/assessment methodology will always beat a 
> Pen Test for the determination of compliance. The issue is 
> that it also requires a far more significant level of skill. 
> A pen test is only 30-35% as effective as a white box audit 
> assessment (assuming both are completed by competent personal).
>
>
>
> A pen test limits the tester making the results less reliable 
> for the benefit of hubris about wanting to do something g 
> cool and be like the 3l1t3. The idea is not if the technique 
> is cool or popular, but what gives the most information to 
> the client. There is still a place for a pen test 
> methodology, but not in most of the examples used in this thread. 
>
>
>
> Regards,
>
> Craig
>
>
> --------------------------------------------------------------
> ----------
> This list is sponsored by: Cenzic
>
> Need to secure your web apps NOW?
> Cenzic finds more, "real" vulnerabilities fast.
> Click to try it, buy it or download a solution FREE today!
>
> http://www.cenzic.com/downloads
> --------------------------------------------------------------
> ----------


------------------------------------------------------------------------
This list is sponsored by: Cenzic

Need to secure your web apps NOW?
Cenzic finds more, "real" vulnerabilities fast.
Click to try it, buy it or download a solution FREE today!

http://www.cenzic.com/downloads
------------------------------------------------------------------------