Nmap Development mailing list archives
[NSE] Robots rethink
From: "Eddie Bell" <ejlbell () gmail com>
Date: Wed, 4 Jun 2008 12:14:50 +0100
Hi all, I've had this new robot script lying around for a while. The current version prints out all disallowed entries in a site's robots.txt file which takes up a lot of screen estate[1]. The attached version just prints a message if a site has a robots.txt file with disallowed entries[2]. What do you think? - eddie [1] 80/tcp open http | robots.txt: /news?output=xhtml& /search /groups /images | /catalogs /catalogues /news /nwshp /? /addurl/image? | /pagead/ /relpage/ /relcontent /sorry/ /imgres < ... this goes on for another 26 lines ... > [2] 80/tcp open http |_ robots.txt: has disallowed entries
Attachment:
robots.nse
Description:
_______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://SecLists.Org
Current thread:
- [NSE] Robots rethink Eddie Bell (Jun 04)
- Re: [NSE] Robots rethink Kris Katterjohn (Jun 04)
- Re: [NSE] Robots rethink Eddie Bell (Jun 04)
- Re: [NSE] Robots rethink jah (Jun 04)
- Re: [NSE] Robots rethink Kris Katterjohn (Jun 04)
- Re: [NSE] Robots rethink Eddie Bell (Jun 04)
- Re: [NSE] Robots rethink Fyodor (Jun 04)
- Re: [NSE] Robots rethink Eddie Bell (Jun 05)
- Re: [NSE] Robots rethink Eddie Bell (Jun 04)
- Re: [NSE] Robots rethink Kris Katterjohn (Jun 04)