Nmap Development mailing list archives
Re: Web App Scanner - GSoC 2009
From: Fyodor <fyodor () insecure org>
Date: Sun, 29 Mar 2009 23:52:53 -0700
On Sat, Mar 28, 2009 at 02:15:17AM -0300, João wrote:
The idea is developing a Web app scanner. Before scanning a host and finding a web server running on it, it would be very interesting that you could have a way to discover which applications are running in this web server.
Hi João! I agree that could be very useful! In fact, I wrote this in my Nmap book: "Nmap may also grow in its ability to handle web scanning. When Nmap was first developed, different services were often provided as separate daemons identified by the port number they listen on. Now, many new services simply run over HTTP and are identified by a URL path name rather than port number. Scanning for known URL paths is similar in many ways to port scanning (and to the SunRPC scanning which Nmap has also done for many years). Nmap already does some web scanning using the Nmap Scripting Engine (see Chapter 9, Nmap Scripting Engine), but it would be faster and more efficient if basic support was built into Nmap itself." --http://nmap.org/book/history-future.html
I mean, we could scan for installations of wordpress, php-myadmin, wikis, web-repos, webmin, OSSIM server, webmail services, and many other applications.
Yep!
I really would appreciate some feedback.
I think it is a very promising idea. If possible, it should be done as an NSE module which scripts can call. If desired performance levels can't be reached in LUA for some reason, it could be made into an NSE C module. And if even that isn't performant enough, I suppose it could be built into Nmap (like RPC scan and the like). But the first two options are preferred for maintainability reasons. Also, there may be many NSE scripts which want to see the results of spidering a web site. There should probably be a way for each script to either see all the pages as they are downloaded, or perhaps the web tree should be saved somewhere (in the filesystem or a database or maybe a limited tree in RAM) so that scripts can look through them. Some things which need to be considered and/or discussed: o Performance is obviously critical. Think about how you can make the best use of web technology such as pipelining, keepalive, head requests when you don't need page content, etc. At the same time, it should be configurable so you can say things like "don't request more than 1 URI per second" in order to avoid floodign a web server. o Some scripts only check for URLs (such as app names), whereas others such as SQL injection detection scripts need the page content. You need to think about how to handle that. Maybe the library can have a list of interested scripts and pages to each of them when they are retrieved. Or maybe you need to store the whole web tree (with some limits so you don't download gigabytes by default or get stuck in infinite loops with dynamically generated pages) o You sent your ideas to the Umit list too. Note that if it is done in Nmap/NSE, Umit can make use of the functionality too. o We already have some web code available (the url module). It probably makes sense to enhance this rather than start over. o High speed web authentication brute force cracking might be welcome too. o I love the idea of fingerprinting web applications, though you'll need to think about how to collect the DB of URLs, how to deal with URLs which can be relocated (in a different directory, say), how to deal with determining the right name for the "host" header, etc. o I'm wondering if it might make sense to apply as some sort of NSE Web maven. Then you could have a list of concrete web-related NSE goals. These might take the form of improvements to the NSE infrastructure, module updates, and/or new scripts. Cheers, -F _______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://SecLists.Org
Current thread:
- Web App Scanner - GSoC 2009 João (Mar 27)
- Re: Web App Scanner - GSoC 2009 Patrick Donnelly (Mar 27)
- Re: Web App Scanner - GSoC 2009 Fyodor (Mar 29)
- Re: Web App Scanner - GSoC 2009 João (Mar 31)
- <Possible follow-ups>
- Re: Web App Scanner - GSoC 2009 Rob Nicholls (Mar 28)
- Re: Web App Scanner - GSoC 2009 João (Mar 28)
- Re: Web App Scanner - GSoC 2009 Fyodor (Mar 30)