(SiteGenerator) re: benchmarking the web app scanners

[Dinis Cruz <dinis.cruz () googlemail com>]

Thomas, Thanks for you kind comments :)

I was going to wait a little bit before I posted this (
http://owasp.net/forums/428/ShowPost.aspx) information here
(webappsec () securityfocus com) , but since the genie is now out of the
bottle, here are a series of quick comments:

1) this tool has been sponsored by Foundstone, BUT (and it is a bit
but) it is being released under the Owasp .Net Project and an Open
Source Licence. So Kudos for Foundstone for doing this and I hope they
get good exposure from it

2) The main objective of the tool is to create dynamic websites based
on XML files which will 'map' to a database containing hundreds of
different vulnerabilities (some simple to detect/exploit, some harder)
covering multiple languages and web development architectures (for
example navigation: Html, JavaScript, Flash, Java, etc...)

3) There are many ways this tool can be used, here are just a couple
starting ideas:
    a) As a training tool since it allows the creation of multiple
websites with multiple variations of vulnerabilities
    b) As a Web Application Honeypot (since we are able to create
dynamic ( i.e. false) websites and track / monitor in real-time all
requests made)
    c) As a test ground for newly discovered vulnerabilities types and
its exploit vectors
    d) As a benchmark for Web Security Scanners

4) The Web Security Scanner benchmarking and testing is the most
obvious short-term application for this tool, but I think that as it
evolves the others will be proven to be as (if not more) valuable

5) On the Web Security Scanner issue:

    a) My main hope is that the Web Security Scanner Companies will
see this tool as an opportunity and work with the Owasp .Net project
(and other groups that want to be involved) in a productive and
constructive way.

    b) Although in the short term some Web Security Scanners might
have some bad results (well, at least when compared with what their
Marketing machine publishes :) in the medium term, as they adapt and
improve their scanning techniques, everybody will benefit

    c) One of the core objectives of the tool (when thinking about
benchmarking Web Security Scanners) is to be able to create real and
measurable metrics. For example:
        - Scanner X was able to detect 65% of the vulnerabilities
where Scanner Y was able to detect 90%
        - Scanner X made 10000 to detect those 65% (over a period of
16h) where scanner Y made 4000 request (over a period of 10h)
        - 20% of Scanner X results where false positives, where
Scanner Y had 50% false positives
        - Scanner X was able to deal with Html and JavaScript
navigation, Scanner Y was able to deal with Html, JavaScript and
Flash, and both where NOT able to deal with Java based navigation
systems
        - Scanner X is not able to go more than 40 levels deep,
Scanner Y is able to go up at least 100 levels deep (if note more)
        - etc, etc, etc.

    d) There will be two main types of tests that can be done in the short term:
        i) provive the links to all different types of vulnerabilities
existent in the database, and see how many can the scanner correctly
identify? and
        ii) when multiple types of website architectures and
navigation techniques are used, how many vulnerabilities is the
scanner able to detect?

    e) In order to test (and further improve the tool) I want to take
this opportunity to ask the Web Application Security Scanners that
subscribe to this list (which I believe all do) to give the Owasp .Net
project a temporary licence to they product so that we can use it
during  development and during some basic benchmarking that we might
do (and NO, I will not sign an NDA that doesn't allow me to publish
the data collected, in fact I will not sign ANY NDA with ANY web
application security scanner company)

    f) note that at the moment I (Dinis) have no plans to do a full
benchmarking exercise since I don't have the time required, but I know
of at least one group of experienced security consultants which is
starting such project (and I will be supporting them). If anybody else
is interested in doing a similar benchmarking project please contact
me directly

6) Regarding how the tool works, here is a brief technical description:

    There are two main components: A webserver (which can be IIS or a
custom webserver) and a GUI application (written in C# 2.0). The Gui
Application is responsible for handling all mappings (from the virtual
requests to the actual pages on disk) and there is an unmanaged C++
DLL loaded by both which implements a Shared Memory to send and
receive data between them.

    The current version is hardcoded to IIS, although in the code
there is support for using a custom .Net webserver. This IIS version
uses an HttpHander to capture all requests and communicate with the
GUI Application (called SiteGeneratorGUI). The previous version used
C++ Detours to hook all sorts of functions in either IIS or the Custom
Webserver (this worked ok, but ultimately I decided to use IIS since
it was much more robust and scalable)

    The dynamic websites are defined by XML files like this (which are
edited on the GUI Application using the WYSIWYG Altova Authentic
Browser Object (SPS files created via Altova's StyleVision
application)):

     <?xml version="1.0" encoding="utf-8" ?>
    <SiteGenerator name="SiteGenerator Demo"
xmlns:ipo="http://www.altova.com/IPO"; xmlns="
http://www.xmlspy.com/schemas/orgchart";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";>
         <site>
             <folder name="">
                <file mappedTo="aspx/Default.aspx" name="HelloWorld.aspx" />
                <folder name="htm" />
                <folder name="aspx">
                    <file mappedTo="aspx/pages.htm" name="pages.htm" />
                    <file mappedTo="aspx/xss.aspx" name=" xss.aspx" />
                    <file mappedTo="aspx/SqlInjection_Easy.aspx"
name="SqlInjection.aspx" />
                    <file mappedTo="aspx/SqlInjection_Hard.aspx"
name=" SqlInjection2.aspx" />
                </folder>
                <folder name="flash">
                     <file mappedTo="flash/cromas_xml.swf"
name="cromas_xml.swf" />
                     <file mappedTo="flash/cromas_xml.htm" name="menu.htm" />
                     <file mappedTo="/flash/cromas_menu.xml"
name="cromas_menu.xml" />
                </folder>
             </folder>
        </site>
    </SiteGenerator>

    SiteGeneratorGUI.exe and IIS will map the virtual name
"HelloWorld.aspx" to the file on disk "aspx/Default.aspx" . For
example:

        http://localhost/HelloWorld.aspx --> F:\Owasp
SiteGenerator\SiteGenerator_ContentPages\aspx\Default.aspx

    So to create new websites all you need to do is to create a new XML file

    And to create new vulnerabilities type, all you need to create in
an Aspx page and map it to the xml file

7) the tool is still in Beta, so please be patient with it. The code
is still a bit in mess, since there are multiple past experiments in
there which I will need to clean up

8) What we need now is:
    a) comments on how the tool works
    b) new vulnerabilities (i.e. new Aspx pages)
    c) help in the development of the missing features (namely an Xml
Metadata section to contain details about each vulnerabilities (risk
rating, remediation, exploit difficulty level, etc...). I also want to
add support for other types of languages and servers (at the moment it
is limited to .Net Framework and IIS (I had ASP Classic in one of the
earlier betas, but didn't include it in this one since it was
implemented using the detours technique)). I have many more ideas on
what is comming next, but first I want to make sure that what this
version is fully functional and usable

I hope this make since (it's 1:30am here and I just wanted to do a
quick brain dump with some ideas and pointers about SiteGenerator)

Best regards

Dinis Cruz
Owasp .Net Project


On 20/01/06, thomas.jones () hushmail com < thomas.jones () hushmail com> wrote:

This tool looks awesome. About time someone built this! Has anyone
got a plan in place to provide some independent, objective results
rather than the marketing hype I have to read from the likes of
eWeek?

Dinis you rock !

http://owasp.net/forums/428/ShowPost.aspx

-------------------------------------------------------------------------
This List Sponsored by: Watchfire

Watchfire's AppScan is the industry's first and leading web application
security testing suite, and the only solution to provide comprehensive
remediation tasks at every level of the application. See for yourself.
Download AppScan 6.0 today.

https://www.watchfire.com/securearea/appscansix.aspx?id=701300000003Ssh
--------------------------------------------------------------------------