RE: (webrute) How to list all the URLs on a web server

["Evans, Arian" <Arian.Evans () fishnetsecurity com>]

Sakaba,

Lyal is correct,

http://www.cirt.dk/tools/webrute.txt is what you need
based upon your problem description.

Perl scripts with regex strings, essentially what webrute
is doing, will work as well.

If the application accesses the files *via* the webserver,
the filenames will likely be in the logs, as most webservers
log full URL by default.

I have seen this before as well, where data was batched
in files--either due to memory use limitations or failure
to understand delegation of privilege on the MS platform.

In either case, it's an insecure practice.

If there is unauthenticated write access, one can also
demonstrate file mod, file deletion, and DoS by filling
all available disk through arbitrary file upload.

Based upon your scenario description, many of the other
tools suggestions (e.g.-Nikto, Paros, etc.) will not
help you w/out customization, and then you might as well
just use a scripting language with strong HTTP support
and write your own queries.

Arian

> -----Original Message-----
> From: Lyal Collins [mailto:lyal.collins () key2it com au] 
> Sent: Friday, January 07, 2005 5:46 PM
> To: 'Lists'; webappsec () securityfocus com
> Subject: RE: How to list all the URLs on a web server
>
> Webrute.pl, by Dennis Rand, may be able to help here.
> I tried this on a test machine last week. It's noisy in the logs, but
> thorough, and slow with large brute force combinations. Take 
> a look - its at
> cirt.dk, I think.
>
> Lyal.
>
>
> -----Original Message-----
> From: Lists [mailto:sakaba () alexandria cc] 
> Sent: Saturday, 8 January 2005 3:35 AM
> To: webappsec () securityfocus com
> Subject: How to list all the URLs on a web server
>
>
> Hi Everyone,
>
> I am auditing a system where files are stored on a web server and 
> accessed without authentication directly by an application that knows 
> each file URL.  I don't like it but the app owner wants me to 
> demonstrate that someone could guess the URLs.  I have tried a number 
> of spider tools but they are based on links so they don't pull up 
> anything.
>
> I am wondering if there is a tool or another method where I 
> could find 
> out all the URLs on the web site.   The funny thing is I saw 
> this same 
> kind of system with the same explanation just the other week 
> at another 
> company.  Maybe its a new trend...
>
> Regards,
> sakaba


The information transmitted in this e-mail is intended only for the addressee and may contain confidential and/or 
privileged material. 
Any interception, review, retransmission, dissemination, or other use of, or taking of any action upon this information 
by persons or entities
other than the intended recipient is prohibited by law and may subject them to criminal or civil liability. If you 
received this communication 
in error, please contact us immediately at 816.421.6611, and delete the communication from any computer or network 
system.