Penetration Testing mailing list archives
Re: [PEN-TEST] Evaluating Auditors Abilities
From: Steve <steve () SECURESOLUTIONS ORG>
Date: Thu, 7 Sep 2000 10:33:47 -0600
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I hate to say it but I agree with some of Derrick's findings. In a past employment situation, I found the company (I won't name them but they are a big 5) that we used for IT Security Audits spend more time/money on things that turned out to be non-issues. I even saw one case where some "kid" walked into a banking client armed with a copy of a popular scanning tool, ran the scan, printed the report and handed it over along with a bill. Ask him to explain the report of the false positives and he couldn't. Its clear that companies need to do their homework before hiring someone to do an audit. I would recommend looking at the potential auditor as a whole, who they employ, and what discoveries/advisories they have released. See if they can provide references and don't be fooled by the marketing machines of the bigger companies. Some of the most talented people work for smaller organizations. Regards; Steve Manzuik Moderator - Win2K Security Advice Security Analyst - Bindview RAZOR http://razor.bindview.com - -------------------------------------------
-----Original Message----- From: Penetration Testers [mailto:PEN-TEST () SECURITYFOCUS COM]On Behalf Of Derrick Sent: Wednesday, September 06, 2000 10:46 PM To: PEN-TEST () SECURITYFOCUS COM Subject: [PEN-TEST] Evaluating Auditors Abilities Dear Pen-Testers, Recently I underwent something that had me thinking about Security Auditing companies and others (Big accounting firms that offer a side service of auditing). Management decided that we needed to be audited by an outside firm, which I am in full favor of. The problem came about in what an un-named auditor did. Firewalls tend to cause false positives in some tests and other anomalies that many auditors may not be aware of. So they performed this audit which we did pick up and were aware of. What happened next is what baffles me. The auditors did not understand the results that nmap and other tools gave them. Near the end of the business day they contact management proclaiming they have found numerous security issues and even some backdoors in our network. After a long couple of days of testing we found none of these issues were correct, and we then spent many hours and several meetings explaining that the firm hired didn't seem to know what they were doing. Management made the default comment of "We are paying them a lot so they must be right, fix these problems". After several days of explaining why they results were wrong and verifying the network we came out to show that the auditors did in fact improperly interpret the results. The end result is management walks away wondering if they got ripped off or if we were just trying to cover problems. It also caused a lot of overtime and extra work for us to explain and prove the network to management. So the end questions are these. How can companies decide which auditors really do a decent job and are worth their value ? Are there any certifications or Industry groups out there or on the horizon that will evaluate and endorse auditors ? What is the best approach from a Network Admin position to counter end results delivered by auditors if they seem to be in error ? Has anyone else been through this, and is destined to get worse before getting better ? Thanks for any thoughts or comments,
Derrick -----BEGIN PGP SIGNATURE----- Version: PGPfreeware 6.5.3 for non-commercial use <http://www.pgp.com> iQA/AwUBObfDYDV9eGvIXwM6EQLXZwCgjzE1SO+CcaklrnMyk1jX50Yx0NoAoKPh 7R/mSdnmZDnm2qmRI2xDoSu3 =iSZH -----END PGP SIGNATURE-----
Current thread:
- Re: [PEN-TEST] Firewall identification and penetration Mike Ireton (Sep 02)
- Re: [PEN-TEST] Firewall identification and penetration Ben Lull (Sep 06)
- [PEN-TEST] Evaluating Auditors Abilities Derrick (Sep 07)
- Re: [PEN-TEST] Evaluating Auditors Abilities Steve (Sep 07)
- Re: [PEN-TEST] Evaluating Auditors Abilities Domenico De Vitto (Sep 07)
- Re: [PEN-TEST] Evaluating Auditors Abilities Teicher, Mark (Sep 07)
- Re: [PEN-TEST] Evaluating Auditors Abilities Max Vision (Sep 08)
- Re: [PEN-TEST] Evaluating Auditors Abilities Deri Jones (Sep 08)
- [PEN-TEST] Evaluating Auditors Abilities Derrick (Sep 07)
- Re: [PEN-TEST] Firewall identification and penetration Jeffrey Denton (Sep 07)
- Re: [PEN-TEST] Firewall identification and penetration Gary E. Miller (Sep 07)
- Re: [PEN-TEST] Firewall identification and penetration Ben Lull (Sep 06)