Penetration Testing mailing list archives
RE: rules of engagement scope
From: "Omar A. Herrera" <omar.herrera () oissg org>
Date: Sun, 21 May 2006 22:33:47 +0100
Hi, my answers below...
-----Original Message----- From: mr.nasty () ix netcom com [mailto:mr.nasty () ix netcom com] Ivan Arce is correct. "The original author (Mr. Nasty) equated defining the scope of a penetration test to committing (or attempting to commit) fraud on the basis that if you define a precise scope then you are purposely leaving out things that may be important to the general public (I am assuming that he intended to apply that rational to government,public service organization and public companies). So you are talking about a different thing: Fraud (or is it phraud?) ommitted by the penetration tester because she exceed the scope of what she was allowed to do, whereas Mr. Nasty proposed that having a scope defined by the organization subject to the test is somehow equivalent to fraud (if the results of the test are not made public)" The only rational that I can see from what Ivan's written is that he has been there. Most others have not. That's why there is a complete disconnect between logic and reason.
So, should we assume that by recognizing that Ivan has been "there" you are at least taking his comments more seriously? (I would, he is a well respected, well known, information security professional with a lot of experience). If that's the case, what is your opinion on these comments that were also posted by Ivan in the same email that contained the comments that you quoted above? : " I submit that scope definition prior to a penetration test is a good thing because it syncs both the tester and the testee on what is considered important, valid, desired, etc., ant helps to plan resource allocation accordingly and to understand and align expectations. BTW you can still define the scope as: "Anything goes, no restrictions whatsoever" but then you would be letting the penetration tester do whatever she feels like doing and unless both parties have a good and long standing relationship it becomes harder for both to assess the costs and the value of the work. " Do you agree or disagree?
Since I receive information on specific audit requirements here is the most recent from ISACA; The Standards Board has issued the following IS Auditing Standards, which become effective for IS audits commencing after 1 July 2006: . S12 Audit Materiality . S13 Using the Work of Other Experts ***** . S14 Audit Evidence My concerns with ROE's are defined within S13. Any big 4 or maybe big 3 now, manager should know this. Audit Managers are brought to the back room by the CFO or CEO presented a pentest within the past 12 months that covered dialup issues. The Everyone smiles and the Audit Manager is lead out of the room with the cover letter stating that the pen-test performed was in conformance with all ROE. The Audit Manager, knowing he has to cut costs or it's coming out of his budget, will accept the pen-test as support and reduce the confidence sample.
Yes, ISACA sends notice on drafts to its members and I also got that one. No, I'm not an auditor anymore and I do "suffer" audits as well as a security officer, just like you, so don't think of yourself as a unique, lucky human being just because you get information from ISACA. Yes, any audit manager should be aware of this stuff, even if they don't work for a big firm. What's the big deal? Yes, these standards have not yet become effective. Yes, many of us are concerned about how vague some of these drafts are and that's why we send comments to ISACA on the drafts before they are approved and sometimes even after. And we are all aware of the Disclaimer section contained in each of these documents, specifically the part talking about "minimum level of acceptable performance". Aren't we? And yes, things like the one you tell us do happen. So what? Do you believe that even with the vague definitions in S13 this action would comply with S13? Is there evidence this would pass the criteria specified by point 06 of this standard (and I make special emphasis on the requirement that this external work must be considered "complete"). The fact that a certain auditor is potentially being negligent is not an excuse for asking that every auditor out there refuses to accept any pentest or any other security assessment with a precise scope. So, according to Ivan's clarification that you quoted, and since you seem to have agreed with that, let me ask you: 1) What proof do you have that by removing any precise scope (i.e. allowing the pentester to test all an everything) will guarantee that all "things that may be important to the general public will be covered"? (Read Ivan's comments on a loose scope). 2) What proof do you have that a complete and detailed pentest (as detailed and complete as you can imagine) can be completed within a reasonable time for any organization? (Don't forget multinational corporations that have offices and systems in most countries of the world and institutions owning class B ip address ranges, those get audited too). Well at least I would expect them to be completed within a year. 3) Have you ever done or even attempted such and extremely detailed and complete pentest yourself with a huge corporation? This is what you are asking for, isn't it? 4) Wouldn't a very light (i.e. non-detailed) yet broad pentest without any precise scope, covering every single thing that you, the stakeholders the general public and every human being on this planet might think of being important in relation to a big organization, fit your requirement to avoid fraud under this definition? After all, scope does not only restrict the things checked, but also how detailed are the tests on these things required to be. If there is no scope, how would you compare if the level of detail was adequate if there is no metric? Wouldn't that metric/threshold be considered as a "precise" scope as well? (Remember the worm in the apple that you mentioned...). Regards, Omar Herrera ------------------------------------------------------------------------------ This List Sponsored by: Cenzic Concerned about Web Application Security? Why not go with the #1 solution - Cenzic, the only one to win the Analyst's Choice Award from eWeek. As attacks through web applications continue to rise, you need to proactively protect your applications from hackers. Cenzic has the most comprehensive solutions to meet your application security penetration testing and vulnerability management needs. You have an option to go with a managed service (Cenzic ClickToSecure) or an enterprise software (Cenzic Hailstorm). Download FREE whitepaper on how a managed service can help you: http://www.cenzic.com/news_events/wpappsec.php And, now for a limited time we can do a FREE audit for you to confirm your results from other product. Contact us at request () cenzic com for details. ------------------------------------------------------------------------------
Current thread:
- Re: rules of engagement scope, (continued)
- Re: rules of engagement scope Ivan Arce (May 17)
- Re: rules of engagement scope Michael Sierchio (May 18)
- Re: rules of engagement scope Ivan Arce (May 18)
- Re: rules of engagement scope Sol Invictus (May 18)
- Re: rules of engagement scope Ivan Arce (May 17)
- Re: Re: rules of engagement scope mr . nasty (May 16)
- RE: Re: rules of engagement scope Omar A. Herrera (May 17)
- Re: rules of engagement scope Marco Ivaldi (May 17)
- Re: rules of engagement scope (DoS testing) Martin Mačok (May 18)
- Re: rules of engagement scope Hylton Conacher(ZR1HPC) (May 21)
- Re: rules of engagement scope mr . nasty (May 21)
- RE: rules of engagement scope Omar A. Herrera (May 23)