Re: spidering of webapps

["Andre Gironda" <andreg () gmail com>]

Burp Spider is the best option.  It is extremely powerful and can be
extended if necessary.  The balance between manual guiding (with forms
and authentication - especially when using the full Burp Suite and
passing information between panels) and automation is near perfect.

That being said, wget does support SSL and for mirroring it works
really, really well.  The --html-extension flag helps with CGI, PHP,
and other non-html file conversion, and the --convert-links makes it
easy to access the content locally for later inspection, potentially
automated.

Cheers,
Andre


On 10/3/08, natron <natron () invisibledenizen org> wrote:
> The unfortunate fact of virtually all local proxies (Burp, Paros, etc)
> is that while, yes, they can do spidering, they have no way to
> save/export results!
>
> HTTrack works, but it lacks on the analysis side, requiring you to do
> a lot of manual reviews of the downloaded files.  I end up relying
> mostly on Burp Suite and just tackling the application in small
> sections and living with the fact that I can't document very well.
>
> Does anyone have any better solutions?
>
> N
>
> On Wed, Oct 1, 2008 at 8:35 PM, Ivan . <ivanhec () gmail com> wrote:
> > Burp Suite
> > http://portswigger.net/suite/
> >
> > Paros
> > http://www.parosproxy.org/download.shtml
> >
> > just a smaple, plenty more out there
> >
> > cheers
> > Ivan
> >
> > On Thu, Oct 2, 2008 at 4:51 AM, <lister () lihim org> wrote:
> > > Other than wget and HTTrack, what other means are you using to
> > > spider/mirror websites?
> > >
> > > How are you spidering through SSL?  OpenSSL wrapper such as stunnel?
> > >
> > > ------------------------------------------------------------------------
> > > This list is sponsored by: Cenzic
> > >
> > > Top 5 Common Mistakes in
> > > Securing Web Applications
> > > Get 45 Min Video and PPT Slides
> > >
> > > www.cenzic.com/landing/securityfocus/hackinar
> > > ------------------------------------------------------------------------
> >
> > ------------------------------------------------------------------------
> > This list is sponsored by: Cenzic
> >
> > Top 5 Common Mistakes in
> > Securing Web Applications
> > Get 45 Min Video and PPT Slides
> >
> > www.cenzic.com/landing/securityfocus/hackinar
> > ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------
> This list is sponsored by: Cenzic
>
> Top 5 Common Mistakes in
> Securing Web Applications
> Get 45 Min Video and PPT Slides
>
> www.cenzic.com/landing/securityfocus/hackinar
> ------------------------------------------------------------------------

------------------------------------------------------------------------
This list is sponsored by: Cenzic

Top 5 Common Mistakes in 
Securing Web Applications
Get 45 Min Video and PPT Slides

www.cenzic.com/landing/securityfocus/hackinar
------------------------------------------------------------------------