Penetration Testing mailing list archives
Spidering
From: "me me" <securityoneoone () googlemail com>
Date: Thu, 17 Jan 2008 17:58:41 +0000
Hi All One of the most important aspects of assessing a web application is separating the dynamic code from the static HTML. I tend to do this though spidering, and use a number of tools including Paros and a commercial tool. Basically, I like to know: The names and places of the scripts The variables they take The Comments etc in HTML Hidden form values All the usual. Whilst I don't expect it to get everything (JavaScript etc is going to take manual intervention, so is a number of other possible technologies), I have never really found a tool that I consider to be the defacto spidering tool from this perspective. One of the biggest problems is a lot of the spiders seem to choke on really big sites, or go into infinite loops etc etc. Has anyone got a reliable spider that they've used time and again (even on big sites) that they could recommend? Or are most people WGETing the site and then regex'ing the local mirror? Thanks in advance ------------------------------------------------------------------------ This list is sponsored by: Cenzic Need to secure your web apps NOW? Cenzic finds more, "real" vulnerabilities fast. Click to try it, buy it or download a solution FREE today! http://www.cenzic.com/downloads ------------------------------------------------------------------------
Current thread:
- Spidering me me (Jan 18)
- RE: Spidering Thor (Hammer of God) (Jan 22)
- Re: Spidering Tonnerre Lombard (Jan 22)
- RE: Spidering merigoth (Jan 23)
- <Possible follow-ups>
- RE: Spidering thomas chamberlain (Jan 23)
- Re: Re: Spidering metabolic_76 (Jan 23)