Re: Web Application Source Vulnerability Scanners

["Kevin Spett" <kspett () spidynamics com>]

Moderator: As SA stated, this is delicate as it involves the discussion of
commercial software produced by the companies that I (and Ory) work for.
However, I think that my points are valid and discussion-worthy.

First, there are other tools besides AppScan that know how to keep state
correctly.  WebInspect does.

Second, on to the aforementioned article.  The info security magazine test
report is less than scientific, as it doesn't detail any of the exact
testing procedures, server configuration, give source code for the
applications, etc.  It does not allow anyone to duplicate or evaluate its
findings.  Also, the test was financed and performed by a company that makes
money by performing services that tools such as AppScan and WebInspect are
designed to test.  Finally, the two authors of the article are not
well-known in the area of web application security.

Keep in mind that these are not accusations.  I am not alleging that the
test results were incorrect.  I am not saying that the authors are
unqualified.  I'm just saying that the test really doesn't really provide
enough information for real technical discussion.  Thus, its findings cannot
be "proved" either way.  I distrust that which cannot be proved.

I simply recommend that people who are interested in appraising the quality
of web security tools, both free or commercial, make their own tests and
judgements, so that they can control every variable of the analysis.  This
will have to be the case until there are truly "open" evaluations that are
not lacking in steps for reproduction.



Kevin Spett
SPI Labs
http://www.spidynamics.com/


----- Original Message -----
From: <securityarchitect () hush com>
To: <webappsec () securityfocus com>; <ory.segal () sanctuminc com>
Sent: Tuesday, March 04, 2003 11:48 AM
Subject: RE: Web Application Source Vulnerability Scanners


> I know this list doesn't cater for commercial tool discussions
specifically so choosing words carefully moderator ;-)
> To counter that you should look at the latest review of commercial tools.
All failed pretty miserably and the general recomendation was to wait until
the next generation of tools come out.
> http://www.infosecuritymag.com/2003/jan/cover.shtml
>
>
> On Tue, 04 Mar 2003 07:25:02 -0800 Ory Segal <ory.segal () sanctuminc com>
wrote:
> > Hi,
> >
> > The problem with most open source tools is that they are very strong
> > in
> > CGI Scanning, but when it comes to mutating real HTTP requests,
> > and
> > testing the web application layer, they lack good engine features.
> > They
> > do not have features such as:
> > 1) Application level tests such as manipulation of : HTML form
> > parameters (SQL Inj., Buffer Overflows, Poison null byte, Format
> > strings
> > bugs, Cookies, HTTP Headers etc...)
> > 2) Automatic testing validation.
> > 3) Good reporting abilities
> > 4) Session management/Transient management - Keeping the scanner
> > 'in
> > session'. This gives you the ability to scan web applications that
> > force
> > you to login, and may kick you out of session, if you caused some
> > error
> > - I believe that most large web apps have this. I believe that AppScan
> >
> > is the only scanner to perform this action.
> > 5) Good performance
> > 6) Contstant updates.
> > 7) Logging of raw HTTP traffic
> > 8) The ability to easily implement new tests.
> >
> > -Ory Segal.
>
>
>
> Concerned about your privacy? Follow this link to get
> FREE encrypted email: https://www.hushmail.com/?l=2
>
> Big $$$ to be made with the HushMail Affiliate Program:
> https://www.hushmail.com/about.php?subloc=affiliate&l=427