Penetration Testing mailing list archives
Re: [PEN-TEST] Evaluating Auditors Abilities
From: "Gallicchio, Florindo (2282)" <florindo.gallicchio () ESAVIO COM>
Date: Thu, 7 Sep 2000 18:11:38 -0400
I certainly agree that it's a good idea for the client to ask the right questions and to determine if the auditing process will work for them. However, there aren't many clients out there who know what the right questions are to evaluate the auditors' capabilities. Most clients don't understand the audit process to begin with, so they don't have a frame of reference from which to base their questions. We give our clients a thorough explanation of how we will approach the test, including our methods, tools and our deliverables. We find this to be a great selling point since many of our competitors don't reveal this much information during the presales phase. We then offer as a part of the audit contract on-the-job training for our clients as we are conducting the audit. It's a "side-saddle" training where we teach the client our methodology and the fundamentals (security principles) behind the audit. This allows the client to better understand what we're doing and gives them some say in how the deliverable will be formed. Florindo Florindo Gallicchio Director, Security Services esavio -----Original Message----- From: Kuss, Kenneth To: PEN-TEST () SECURITYFOCUS COM Sent: 9/7/00 1:36 PM Subject: Re: [PEN-TEST] Evaluating Auditors Abilities It is very hard to assess the abilities of the outside auditors that will be conducting the reviews. You can look at resumes, engage reputable firms, etc. I believe that the most important part of engaging any outside assistance to establish the process up front. All aspects of the review should be covered including; tools to be used, reporting deliverables and the process to verify results of any findings before reporting to upper management. This gives the operators of the system, network, etc. the ability to address any false positives identified. Derrick <Derrick () ANEI COM>@SECURITYFOCUS.COM> on 09/07/2000 12:46:27 AM From: Derrick <Derrick () ANEI COM>@SECURITYFOCUS.COM> on 09/07/2000 12:46 AM Please respond to Penetration Testers <PEN-TEST () SECURITYFOCUS COM> Sent by: Penetration Testers <PEN-TEST () SECURITYFOCUS COM> To: PEN-TEST () SECURITYFOCUS COM cc: Subject: Evaluating Auditors Abilities Dear Pen-Testers, Recently I underwent something that had me thinking about Security Auditing companies and others (Big accounting firms that offer a side service of auditing). Management decided that we needed to be audited by an outside firm, which I am in full favor of. The problem came about in what an un-named auditor did. Firewalls tend to cause false positives in some tests and other anomalies that many auditors may not be aware of. So they performed this audit which we did pick up and were aware of. What happened next is what baffles me. The auditors did not understand the results that nmap and other tools gave them. Near the end of the business day they contact management proclaiming they have found numerous security issues and even some backdoors in our network. After a long couple of days of testing we found none of these issues were correct, and we then spent many hours and several meetings explaining that the firm hired didn't seem to know what they were doing. Management made the default comment of "We are paying them a lot so they must be right, fix these problems". After several days of explaining why they results were wrong and verifying the network we came out to show that the auditors did in fact improperly interpret the results. The end result is management walks away wondering if they got ripped off or if we were just trying to cover problems. It also caused a lot of overtime and extra work for us to explain and prove the network to management. So the end questions are these. How can companies decide which auditors really do a decent job and are worth their value ? Are there any certifications or Industry groups out there or on the horizon that will evaluate and endorse auditors ? What is the best approach from a Network Admin position to counter end results delivered by auditors if they seem to be in error ? Has anyone else been through this, and is destined to get worse before getting better ? Thanks for any thoughts or comments, Derrick
Current thread:
- Re: [PEN-TEST] Evaluating Auditors Abilities Emeigh, Mike (Sep 07)
- Re: [PEN-TEST] Evaluating Auditors Abilities topher hughes (Sep 08)
- <Possible follow-ups>
- Re: [PEN-TEST] Evaluating Auditors Abilities Tansey, Don (Sep 07)
- Re: [PEN-TEST] Evaluating Auditors Abilities Benjamin P. Grubin (Sep 07)
- Re: [PEN-TEST] Evaluating Auditors Abilities Kuss, Kenneth (Sep 07)
- Re: [PEN-TEST] Evaluating Auditors Abilities Edward Slusarski (Sep 07)
- Re: [PEN-TEST] Evaluating Auditors Abilities David Hopkins (Sep 07)
- Re: [PEN-TEST] Evaluating Auditors Abilities Khan, Mansoor (Sep 08)
- Re: [PEN-TEST] Evaluating Auditors Abilities Meritt, Jim (Sep 08)
- Re: [PEN-TEST] Evaluating Auditors Abilities Dunker, Noah (Sep 08)
- Re: [PEN-TEST] Evaluating Auditors Abilities Gallicchio, Florindo (2282) (Sep 08)
- Re: [PEN-TEST] Evaluating Auditors Abilities Hill, Mark (Sep 08)