Firewall Wizards mailing list archives
Re: Penetration testing via shrinkware
From: Dominique Brezinski <dom_brezinski () securecomputing com>
Date: Thu, 03 Sep 1998 12:11:44 -0500
Just so everyone knows my bias up front: I worked for ISS at one point as part of the X-Force (the research group that develops exploit checks for the scanner and ID signatures for RealSecure amongst other things) and I currently work for Secure Computing in the architecture/assessment branch of Professional Services. I do work very closely with our penetration team, and occasionally I actually do some penetration testing too. At 07:22 PM 9/2/98 -0400, Stout, Bill wrote:
What are the opinions on the thoroughness of shrinkwrap software penetration testing? Is today's shrinkware more capable for penetration testing (a single machine) than a human?
Absolutely not. There is a constraint in developing shrinkwrap vulnerability scanners that the code can not really change the configuration of the target machine or have the potential to corrupt or disable running services. Most of the checks in commercial software vulnerability scanners don't actually exploit the problems, rather they *identify* them. There are some vulnerabilities that *must* be exploited to even know they are there, and in a few cases the commercial software scanners will cross the line, so to speak, because it is the only way to identify the problem. What commercial software scanners don't do is actually exploit the machine and systematically rummage through it looking for information that can be used to penetrate other machines (or networks). There are certain isolated instances where the scanners make use of some secondary information, but it is not nearly at the level a human can/will. SATAN did some of this transitive trust following, but again it isn't anywhere near what a human can do. Think of it this way: software scanners tend to do a great job at finding the first layer of vulnerabilities, but a human penetration team can possibly find N (N being affected by many things including time, money, and they security of the tested system). That is why human penetration teams often use software scanners in the initial phase, because they do a good job of pointing out open doors. It is then up to the pen team to go through them. If the penetration team you hire isn't going through those doors, then you are getting ripped off and you should just license the scanner yourself!
I'll take one example of a tool, Internet Security Scanner. It can do a complete external scan of the currently known vulnerabilities of a machine or subnet. ISS is very consciensious of keeping up to date with vulnerabilities.
No matter what, the commercial vulnerability scanners will always be behind the underground community. A good penetration team will have some communication channel with the underground, and hopefully will be aware of exploits that are not in the commercial scanners yet. Actually, penetration teams are often one of the places the commercial scanner companies get there exploit knowledge from. Again, the better penetration teams do their own vulnerability research, and therefore are ahead of the curve in some regards. Vulnerability research is part of my job. I am expected to add value to our services above and beyond just running commercial software scanners and cleaning up the report (though that is exactly what a lot of "penetration teams" do).
Some counter-points I have are: o The human needs to do data collection about the target through whois, nslookup, search engines, anonymous or spoofed phone calls, etc. o The human element still needs to select the targets, the connection path (dial-up, X.25, Internet, hops via private links, etc), the social engineering, the password crackers, etc. o The human also needs to define the D.O.S. threshold of the target, and limits on brute force efforts. o The tests won't detect sniffers installed at the target's ISP.
Even with the best expert system or artificial intelligence technologies, a software based system can not hack as well as a human (though with a few million and a couple years, we could probably get pretty close ;). Penetration testing pulls in a lot of diverse knowledge and experiences. It is really simple sometimes, but other times it can be extremely complex. It is not uncommon for our team to write new tools or attacks on the spot, because we were able to recognize a vulnerability that consists of series of issues with some previously unknown aspect to it. Believe me, if we could write a tool that did everything we can, I would be writing it right now so we could click a few buttons and all go to coffee - permanently ;)
Say someone wants to do penetration testing and security auditing for a company, and use various types of shrinkware to do it. Any comments?
Shrinkwrap vulnerability scanners *should* be *one* of the tools used in a penetration test, but not the only thing used. If the auditor or penetration team does not bring more to the table than a shrinkware tool kit, then license the same software, run it yourself, and save a ton of cash (actually, see if you can get their consulting rate minus 10% or something ;). IMHO, Dominique Brezinski CISSP (612)628-5378 Secure Computing http://www.securecomputing.com
Current thread:
- Penetration testing via shrinkware Stout, Bill (Sep 03)
- Re: Penetration testing via shrinkware Bennett Todd (Sep 03)
- Re: Penetration testing via shrinkware Sheila //or// Bob (depends on who's writing) (Sep 06)
- Re: Penetration testing via shrinkware Stephen P. Berry (Sep 06)
- <Possible follow-ups>
- Re: Penetration testing via shrinkware Marcus J. Ranum (Sep 03)
- Re: Penetration testing via shrinkware emaiwald (Sep 03)
- Re: Penetration testing via shrinkware Dominique Brezinski (Sep 03)
- Re: Penetration testing via shrinkware Ryan Russell (Sep 03)
- RE: Penetration testing via shrinkware Gary Crumrine (Sep 03)
- RE: Penetration testing via shrinkware Christopher Nicholls (Sep 07)
- Re: Penetration testing via shrinkware tqbf (Sep 17)
- Re: Penetration testing via shrinkware Crispin Cowan (Sep 18)
- Re: Penetration testing via shrinkware Ted Doty (Sep 19)
- Re: Penetration testing via shrinkware tqbf (Sep 19)
- Re: Penetration testing via shrinkware Dave Whitlow (Sep 19)
- Re: Penetration testing via shrinkware Christopher Nicholls (Sep 19)
- Re: Penetration testing via shrinkware Adam Shostack (Sep 20)
- RE: Penetration testing via shrinkware Christopher Nicholls (Sep 07)