Re: Of the three expensive vulnerability scanners

[Daniel <deeper () gmail com>]

I'd just like to point out Sanctums/Watchfire's rather ridiculous
patent registration

http://lists.netsys.com/pipermail/full-disclosure/2004-January/015690.html




On Sat, 13 Nov 2004 16:22:36 -0500, Jim+Lisa Weiler <lisajimbo () rcn com> wrote:
> I've run WebInspect (WI, SPIDynamics) and AppScan (AS, Sanctum, now
> Watchfire) against 2 large ecommerce site code bases, written in IIS/ASP/VB
> and Apache/IBM Websphere - Websphere Commerce Server/JSP. Both products seem
> equally comprehensive. AS is faster and Watchfire says they do support
> better because they are bigger and have a support organization. I found both
> organizations to be responsive. The Watchfire folks didn't seem to know as
> much about the world outside their product as the WI folks did. Another
> dimension that I found very important to evaluate, is how useful the
> reporting is for planning remediation projects and actually fixing the
> problems. Both products might show 100 vulnerabilities. This might actually
> only involve 5 web pages because they count a vulnerability as a single
> failed test on a single field on 1 page. 5 failed XSS tests on the same
> field in the same page will count as 5 vulnerabilities. If the XSS tests are
> in different categories (HTML injection vrs unchecked parameter) they will
> be in different parts of the reports. AS is very poor at telling you
> 1. what are the different unique web pages (no repeated web pages) with a
> vulnerability
> 2. for each vulnerability type, what pages does it occur on
>
> Just finding out exactly what different pages were involved took 2 hours
> looking at the AS report screen. They report the same page over and over
> again in different places.
> WI is somewhat better and I haven't seen a version later than 5 months ago.
>
> The pie charts, executive reports and spreadsheet exports don't help
> planning the work to fix the problem.
>
>
> ----- Original Message -----
> From: "Tom Stracener" <strace () gmail com>
> To: <webappsec () securityfocus com>
> Sent: Sunday, October 10, 2004 2:45 PM
> Subject: Re: Of the three expensive vulnerability scanners
>
> > In-Reply-To: <20041007153115.28058.qmail () www securityfocus com>
> >
> > Hi! I sought to answer this question for myself a while back, so hopefully
> > you'll find my own experiences here useful. First, consider
> > the types of applications and the application environment you will be
> > securing. Depending upon the complexity of the web application you're
> > dealing with, your likely to get quick diminishing returns from the tools
> > you have mentioned. Strong manual testing capabilities are a must, in my
> > opinion, and sadly a lot of commercial apps fall short there.
> >
> > When possible, you should contact the vendors and acquire a demo license
> > in order to get a feel for how a tool actually performs. If that's not
> > available, then you should sit down with the vendors and get a hands on
> > session.
> >
> > SPI Dynamics is very demo friendly. You'll find their people polite,
> > professional, and quick to respond once you download the product. So if
> > you want to take a look at it, just contact Natalie Hinkle
> > <nhinkle () spidynamics com> if you have any questions or run into problems
> > downloading it. Also, if you go this route be sure to download the SPI
> > Toolkit, which includes some manual pen testing utilities.
> >
> > With Sanctum, acquiring a demo was more difficult, I had to speak with
> > the salesperson's manager and then wait a few days, only to be declined.
> > Only after sending an email to their VP Internal Sales together with my
> > resume did I manged to get a demo. You may have better results. Jane
> > Foulkes <jfoulkes () sanctuminc com> is a sales person you can contact over
> > there.
> >
> > Last I checked Scando did not have a demo available at all.
> >
> > I would also strongly encourage you to contact Cenzic and discuss having a
> > look at their up and coming version of Hailstorm 2.0. Its by far the most
> > extensible of the available commercial offerings. The tool provides a nice
> > balance of automated verses manual app spidering, allows you to record and
> > replay complicated HTTP sessions (which they call traversals) and then you
> > can apply different types of security policies as Hailstorm iteratively
> > steps through the web application. You can also create your own policies
> > and have full control over the fault injectors which interrogate the app,
> > as well as types of response conditions you're interested in detecting.
> > This tool shows an incredible amount of promise, so it would probably be
> > in your interest to evaluate it. You can contact Mandeep Khera over there
> > <mandeep () cenzic com> if you're interested finding out more about it.
> >
> > Also, browse the recent archives of this list because your question
> > has surfaced in various forms and you'll be able to find a variety of
> > useful perspectives.
> >
> > --Tom