RE: Environment for testing WebApp Security Scanners

["Mark Curphey" <mark () curphey com>]

One advantage you might have (if you move quickly) is that I just released
it a couple months ago and the as of 3 weeks ago, the scanners didn't have
specific rules for this app.

This is one of the key design concepts in SiteGenerator. The site is
differenet each time so dynamic. 
-----Original Message-----
From: Gerald Quakenbush [mailto:geraldq () mastermindsecuritygroup com] 
Sent: Tuesday, August 08, 2006 9:40 PM
To: webappsec () securityfocus com
Subject: Re: Environment for testing WebApp Security Scanners

See www.quakenbush.com for an App called MasterBugs - it may be useful for
your testing. I wrote the app (which is GNU licensed) as a companion to my
book.

I recently did a bake-off using the top three scanners / appsec toolkits on
the market. As *automated* scanners I was very disappointed. As toolsets in
the hands of a professional they have varying degrees of usefulness.

MasterBugs is "real", ie, it doesn't fake anything. It was originally a
proof-of-concept app for a real software package. I dumbed it down, added a
lot of bugs (some of the most fun I've had programming ...) and I use it as
a teaching tool. It is written in legacy ASP script and requires a SQL
Server v2000. There are hundreds of remotely exploitable vulnerabilities in
it.

All of the automated scanners I tested failed to find even 10% of the flaws.

One advantage you might have (if you move quickly) is that I just released
it a couple months ago and the as of 3 weeks ago, the scanners didn't have
specific rules for this app.

Have fun...

-Gerald Quakenbush
Author of Web Hacker Boot Camp

Dean H. Saxe (dean () fullfrontalnerdity com) wrote:
> Just remember, some vendors put in signatures specifically for apps 
> like this that they know will be tested.
>
> I spoke with some folks at Blackhat last week from one of the tool 
> vendors that admitted they don't find all the vulnerabilities in 
> HacmeBank.  Why?  Because bypassing the login form using SQL injection 
> may require you to throw away the cookies which maintain the login 
> attempts state information.  For instance, if you try SQL injection 
> too many times and fail to login with those attempts, a cookie which 
> controls the remaining number of login attempts will force all further 
> attempts to fail.  Of course, a human would delete the login attempt 
> counter cookie and solve the problem quite simply.
>
> I see this as a major weakness of these types of tools.  Developers 
> store all kinds of crap in cookies, without a good analysis of the 
> cookies by a human, how do we know when deleting this information will 
> adversely affect the application's security?  This is not an edge case 
> where such tools scan.  Its a very commonly revealed vulnerability in 
> my web app pen testing and code review experience.
>
> (FWIW I work for Foundstone.)
>
>
> -dhs
>
> Dean H. Saxe, CISSP, CEH
> dean () fullfrontalnerdity com
> Here in America everything is bought and sold, you can get anything 
> for little bits of gold.
> We'll rape the earth and ruin the air, cut down every tree from here 
> to there.
>      -- Donna The Buffalo "America"
>
>
> On Aug 8, 2006, at 10:25 AM, Brokken, Allen P. wrote:
>
> > You might also consider looking at Foundstone's Hacme suite of sites 
> > as a compliment to site generator.  I've found that in testing 
> > scanners each methodology for building a site works different 
> > "muscles" in the scanner and having a diverse back drop to test 
> > against is important.
> >
> > Also, it has been useful to have someone unfamiliar with the various 
> > sites and scanners being tested do the actual scanning.
> > I've found that having an intimate knowledge of the site and scanner 
> > can boost individual performance drastically and compared to the 
> > same person using different tools on a site they are unfamiliar 
> > with.
> >
> > You might consider bringing in classmates who are technical, but 
> > unfamiliar with the tools / sites to do at least one round of your 
> > testing.  Most of the results/studies I've seen are done by highly 
> > trained professionals.  However, most purchasers of scanners have 
> > not risen to that level yet.  So a study along those lines would be 
> > very useful.
> >
> >
> > Allen Brokken
> >
> > Information Security and Account Management - IAT Services - 
> > University of Missouri -brokkena () missouri edu - (573)884-8708
> >
> > -----Original Message-----
> > From: René Palige [mailto:rwp () gmx de]
> > Sent: Monday, August 07, 2006 3:33 PM
> > To: webappsec () securityfocus com
> > Subject: Environment for testing WebApp Security Scanners
> >
> > Hi!
> >
> > I?m currently working on my bachelor thesis which is about the 
> > development of a testsuite for different Web Application Security 
> > Scanners. My goal is to provide an environment which can be used as 
> > a basis for testing and evaluating the performance of the many tools 
> > already existing.
> > Consequently the main part of my work will be to implement different 
> > types of vulnerabilites in more or less realistic scenarios and with 
> > different characteristics. At the moment I?m planning to use OWASPs 
> > WebGoat as some kind of groundwork.
> > My questions:
> > Which "features" would you consider to be necessary or useful in 
> > this context? And what basic requirements do you see which should be 
> > met? Would it be best to focus on "real-life scenarios"? Or rather 
> > to cover as many aspects of a special class of vulnerabilities as 
> > possible?
> >
> > Thanks in advance,
> > R. Palige
> >
> >
> >
> >
> > --------------------------------------------------------------------
> > --
> > ---
> > Sponsored by: Watchfire
> >
> > Watchfire was recently named the worldwide market leader in Web 
> > application security assessment tools by both Gartner and IDC.
> > Download a free trial of AppScan today and see why more customers 
> > choose AppScan then any other solution. Try it today!
> >
> > https://www.watchfire.com/securearea/appscancamp.aspx?
> > id=701500000008VnB
> > --------------------------------------------------------------------
> > --
> > ----
> >
> >
> > --------------------------------------------------------------------
> > --
> > ---
> > Sponsored by: Watchfire
> >
> > Watchfire was recently named the worldwide market leader in Web 
> > application security assessment tools by both Gartner and IDC.
> > Download a free trial of AppScan today and see why more customers 
> > choose AppScan then any other solution. Try it today!
> >
> > https://www.watchfire.com/securearea/appscancamp.aspx?
> > id=701500000008VnB
> > --------------------------------------------------------------------
> > --
> > ----
>
>
> ----------------------------------------------------------------------
> ---
> Sponsored by: Watchfire
>
> Watchfire was recently named the worldwide market leader in Web 
> application security assessment tools by both Gartner and IDC.
> Download a free trial of AppScan today and see why more customers 
> choose AppScan then any other solution. Try it today!
>
> https://www.watchfire.com/securearea/appscancamp.aspx?id=701500000008V
> nB
> ----------------------------------------------------------------------
> ----


-------------------------------------------------------------------------
Sponsored by: Watchfire

Watchfire was recently named the worldwide market leader in Web application
security assessment tools by both Gartner and IDC. 
Download a free trial of AppScan today and see why more customers choose
AppScan then any other solution. Try it today!
  
https://www.watchfire.com/securearea/appscancamp.aspx?id=701500000008VnB
--------------------------------------------------------------------------



-------------------------------------------------------------------------
Sponsored by: Watchfire

Watchfire was recently named the worldwide market leader in Web
application security assessment tools by both Gartner and IDC.
Download a free trial of AppScan today and see why more customers choose
AppScan then any other solution. Try it today!

https://www.watchfire.com/securearea/appscancamp.aspx?id=701500000008VnB
--------------------------------------------------------------------------