WebApp Sec mailing list archives
RE: Comparison report on web app security scanners
From: "Erwin Geirnaert" <egeirnaert () securityinnovation be>
Date: Wed, 17 May 2006 22:13:24 +0200
There is a compilation of open-source applications called Securibench from Stanford University: http://suif.stanford.edu/~livshits/securibench/ These are all Java based and include WebGoat. So if someone can test these applications that would be great! Or the security tool vendors can use their tools to find vulnerabilities in open-source apps and share that For me the discussion is not about the use of WebGoat or how the tests were executed. I've read Dr. Peine his report in German and it was a greatdocument. Everything was described in detail and how the tests were executed. If you compare his results with the results of Arian Evans, there is a lot of similarity. Remark: if a security assessment tool is unable to keep state to test an application it cannot be used. If you look at AJAX, or Struts taglib, you need state in every request... Ofcourse there are some other interesting articles from eWeek and Infoworld that tested most security assessment tools out there. So google for that if you want more information. Anyway, I will present on Owasp 2006 about "Can (automated) tools test for the OWASP Top 10" so if you have any experience that you want to share, mail me. Regards, Erwin -----Original Message----- From: Ory Segal [mailto:osegal () watchfire com] Sent: dinsdag 16 mei 2006 16:53 To: webappsec () securityfocus com Cc: Holger.Peine () iese fraunhofer de; Bogdan Calin Subject: RE: Comparison report on web app security scanners Hello, I would like to add several important comments to this thread, in behalf of Watchfire: According to tests done in Watchfire's labs, when using AppScan 6.0 SP2 + update 553 on the WebGoat application - AppScan will find 85 links, will create 9557 tests and will eventually find 31 issues (211 different test variants). People who perform such benchmarks against WebGoat should pay attention to the fact that AppScan needs some configuration in order to run successfully on this application - - Explore method should be set to DFS (Depth First) - Scan should be done in Single threaded mode - Path limit should be disabled (no path limit) - Depth limit should be enabled (otherwise one of the lessons gets into an infinite loop) - HTTP Authentication - use guest/guest - Add the "Screen" parameter to the black-list (untested parameters) - Auto-form filler should be enabled (!!!) IMPORTANT: all of the above configuration items existed in AppScan for a long time, these were not added in order to "cook the product" to work properly on WebGoat. In addition I support what Acunetix mentioned, WebGoat and the FoundStone (Hackme) banking applications are poor examples to be testing on. I am also quoting Mark Curphey (OWASP), regarding OWASP's WebGoat project: "That said its (i.e. Foundstone's HackMe bank) not a good benchmarking tool for testing these tools, nor is WebGoat" - taken from: http://seclists.org/lists/webappsec/2005/Oct-Dec/0025.html Thank you very much, -Ory Segal Watchfire -----Original Message----- From: Bogdan Calin [mailto:bogdan () acunetix com] Sent: Tuesday, May 16, 2006 17:10 To: webappsec () securityfocus com Cc: Holger.Peine () iese fraunhofer de Subject: Re: Comparison report on web app security scanners Hello, A few days ago Dr. Holger Peine published a "Comparison report on web app security scanners". For this report he used two web applications: one of them is WebGoat and the other one is a proprietary application which is not public. I don't know anything about this proprietary application but I would like to say that WebGoat is not a good test case for evaluating web scanners. WebGoat is using server side state variables to track user actions. For example, if you want to test the String SQL injection flaw you first need to navigate to the "String SQL injection" section in order to set the proper state of the application. If the application is not in the proper state, the SQL injection test will not work. The application will just ignore your inputs. An automated scanner cannot guess this application behavior, and unless you optimize your scanner for this particular application it will not be able to scan it properly. When the scanner has finished discovering the site structure, WebGoat will be in some unknown state. All tests will be performed while WebGoat is in this state. This is not a common implementation. Because we are offering free audits, we have audited more than 1,000 websites and didn't encountered this kind of implementation. WebGoat is great for learning about web security flaws but I don't think it should be used as a test case for web security scanners. Bogdan Calin Acunetix Ltd. - www.acunetix.com Acunetix Web Vulnerability Scanner ------------------------------------------------------------------------ - Sponsored by: Watchfire Watchfire named worldwide market share leader in web application security assessment by leading market research firm. Watchfire's AppScan is the industry's first and leading web application security testing suite, and the only solution to provide comprehensive remediation tasks at every level of the application. See for yourself. Download a Free Trial of AppScan 6.0 today! https://www.watchfire.com/securearea/appscansix.aspx?id=701300000007t9c ------------------------------------------------------------------------ -- ------------------------------------------------------------------------ - Sponsored by: Watchfire Watchfire named worldwide market share leader in web application security assessment by leading market research firm. Watchfire's AppScan is the industry's first and leading web application security testing suite, and the only solution to provide comprehensive remediation tasks at every level of the application. See for yourself. Download a Free Trial of AppScan 6.0 today! https://www.watchfire.com/securearea/appscansix.aspx?id=701300000007t9c ------------------------------------------------------------------------ -- ------------------------------------------------------------------------- Sponsored by: Watchfire Watchfire named worldwide market share leader in web application security assessment by leading market research firm. Watchfire's AppScan is the industry's first and leading web application security testing suite, and the only solution to provide comprehensive remediation tasks at every level of the application. See for yourself. Download a Free Trial of AppScan 6.0 today! https://www.watchfire.com/securearea/appscansix.aspx?id=701300000007t9c --------------------------------------------------------------------------
Current thread:
- RE: Comparison report on web app security scanners, (continued)
- RE: Comparison report on web app security scanners Mark Curphey (May 18)
- Re: Comparison report on web app security scanners Zaninotti, Thiago (May 18)
- Re: Comparison report on web app security scanners Eoin (May 17)
- RE: Comparison report on web app security scanners Mark Curphey (May 17)
- RE: Comparison report on web app security scanners Bogdan Calin (May 18)
- Re: Comparison report on web app security scanners solutions_PHP (May 18)
- Re: Comparison report on web app security scanners Bogdan Calin (May 18)
- RE: Comparison report on web app security scanners Mark Curphey (May 19)
- WAF learning ability limitation? matt farey (May 19)
- Re: Comparison report on web app security scanners solutions_PHP (May 19)