Re: OS XSS and SQL scanner

["Dean H. Saxe" <dean () fullfrontalnerdity com>]

Here, here, Arian.

Let's see the web app scanner folks go up against a manual pen test  
and code review/threat model on a series of apps.  One caveat: the  
results must be open for review, which means publishing the results  
in an open forum for all to see.

FWIW, I'm a former customer of SPIDynamics.  I have experience web  
app scanners in an enterprise environment along with pen testing and  
code reviews.  I have a good idea how things will shake out:  Web app  
scanners are inexpensive to run but don't find significant numbers of  
vulnerabilities.  Pen tests are a decent measure of security at a  
reasonable cost when performed my talented testers.  Code review &&  
threat model finds the most vulnerabilities at the highest cost when  
performed by talented reviewers.

Will any web app scanner companies actually subject their scanners to  
such a bake off?  If not, how can we trust the marketing material?   
Was Gary McGraw right in calling these tools "badnessometers"?

I'm at BlackHat all week.  Email me and we'll get together and chat.   
I'll be attending the WASC gathering at Shadow Bar tomorrow night.  I  
hope to see some of you there.

-dhs

Dean H. Saxe, CISSP,  CEH
dean () fullfrontalnerdity com
"[T]he people can always be brought to the bidding of the leaders.  
This is easy. All you have to do is to tell them they are being  
attacked, and denounce the pacifists for lack of patriotism and  
exposing the country to danger. It works the same in every country."
    --Hermann Goering, Hitler's Reich-Marshall at the Nuremberg Trials


On Aug 1, 2006, at 2:35 PM, Arian J. Evans wrote:

> > -----Original Message-----
> > From: Mandeep Khera [mailto:mandeep () cenzic com]
> >
> > I am sorry to hear that you perceive some problems with our
> > product. We take pride in being the most accurate product
> > with least amount of false positives in the industry. This
> > has been proven in many bake-offs by customers and
> > independent journalists.
>
> Hate to take this a little off topic, but do you have any facts
> that can support or back up these claims? Any data produced by
> anyone competent that speaks to your "false positives" and also
> your "false negatives"?
>
> I have failed to read a review yet to date that contains useful
> information. So far what I've read varies from useless data
> organized around features like "reflective buttons" (e.g.-the
> Acunetix review posted to this list written by some woman
> who writes windows software articles) to the other extreme
> of uninformed opinion and inability to keep features between
> the products straight (secure enterprise computing review).
> This includes infosec magazine and online reviews, bake-offs,
> and Gartner-style evals. Every one I have read so far is garbage.
>
> Not one covers actual tests run & and the how & why around them.
>
> This situation is no doubt due to the utter lack of skill
> and understanding of the subject on the part of the authors.
>
> However, I think all on this list would welcome information
> of a high-quality nature regarding scanner quality, if you
> have anything like that to point us at.
>
> -ae
>
>
>
>
>
> ---------------------------------------------------------------------- 
> ---
> Sponsored by: Watchfire
>
> Do you test web applications for XSS, SQL Injections, Buffer  
> Overflows,
> Logical issues and other web application security threats? Why not
> automate this work with Watchfire's AppScan, the world's leading
> automated web application scanner. Download AppScan today!
>
> https://www.watchfire.com/securearea/appscancamp.aspx? 
> id=701300000008BP9
> ---------------------------------------------------------------------- 
> ----


-------------------------------------------------------------------------
Sponsored by: Watchfire

Do you test web applications for XSS, SQL Injections, Buffer Overflows, 
Logical issues and other web application security threats? Why not 
automate this work with Watchfire's AppScan, the world's leading 
automated web application scanner. Download AppScan today!

https://www.watchfire.com/securearea/appscancamp.aspx?id=701300000008BP9
--------------------------------------------------------------------------