Nmap Development mailing list archives
Re: [NSE] Two new scripts: http-google-email and http-reverse-ip
From: Martin Holst Swende <martin () swende se>
Date: Sun, 31 Jul 2011 19:52:33 +0200
On 07/28/2011 01:08 PM, Patrik Karlsson wrote:
On Jul 27, 2011, at 11:01 PM, Shinnok wrote:Hi nmap-dev, As stated in some of my previous e-mails, I started tinkering with NSE and LUA a bit, just so I don't miss this part of Nmap fun and I want to share with you two scripts I've written: 1. Google Web Search and Google Groups Search e-mail discovery This one attempts to search for e-mails pertaining to a specific domain in Google's Web search engine(google.com) and Google Groups search engine(groups.google.com). Sample output: ./nmap -p80 --script http-google-email insecure.org Starting Nmap 5.59BETA1 ( http://nmap.org ) at 2011-07-27 23:53 EEST Nmap scan report for insecure.org (74.207.254.18) Host is up (0.19s latency). rDNS record for 74.207.254.18: web.insecure.org PORT STATE SERVICE 80/tcp open http | http-google-email: | nmap-dev () insecure org | nmap-svn () insecure org |_fyodor () insecure org Nmap done: 1 IP address (1 host up) scanned in 4.01 seconds 2. Reverse domain ip lookup using Bing This one attempts to find domains that are hosted on a specific ip address using Bing's ip: operator. Sample output: ./nmap -p80 --script http-reverse-ip insecure.org Starting Nmap 5.59BETA1 ( http://nmap.org ) at 2011-07-27 23:55 EEST Nmap scan report for insecure.org (74.207.254.18) Host is up (0.20s latency). rDNS record for 74.207.254.18: web.insecure.org PORT STATE SERVICE 80/tcp open http | http-reverse-ip: | nmap.org | insecure.org | secwiki.org |_images.insecure.org Nmap done: 1 IP address (1 host up) scanned in 1.75 seconds The both take a "pages" argument that specifies the number of results pages to request from search engines(the default is 5) and a domain and host argument respectively, that can be used to query a different target then the one nmap scans. I used http-google-malware.nse as a reference and the results are far from perfect and since they are my first attempts at NSE I am welcome to suggestions, improvements and conformance hints. Thanks, Shinnok <http-google-email.nse><http-reverse-ip.nse>_______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://seclists.org/nmap-dev/Hi Shinook, First off, welcome to the great world of NSE :) I've had some experience in the past with "wrapping" google search with tools such as wget. What usually happens after a bunch of queries is that Google block the search possibility from that IP or redirect you to a captcha site or a site making heavy use of javascript to prevent further "automated searches". In the past you were able to get a google API key to get around this problem and use a dedicated 3rd party API that wouldn't impose these restrictions. I'm not sure how this works now. I also think Bing had some similar restrictions requiring an API key. I know that Martin has done some work in this area and may have some newer information. //Patrik
Hi! I've done some work using the API:s available for Bing, Yahoo and Google. I saw that the scripts by Shinnok does not use the API:s, but the regular search functionality. That may certainly work, but there may also be issues. I know that google does its best to stop automated searches using the normal querying facilitites, it wouldn't surprise me if Yahoo/Bing did the same. There are also other issues, from what I understand owasp efforts in creating tools for automated google searches were aborted due to the fact that performing automated searches may be some kind of violations against the TOS. But I don't know any deatails about it, so don't take my word for it. If one wants to do it the 'proper' way, all three have great json api's that they want applications to use. All three used to require API keys, but google later abandoned this approach and opened it up. But not completely - I don't have any links to it, but there is an open issue with it that it does not deliver the same search results as the real thing, and that it the search results are capped (you just won't get everything!). I implemented three apps in python GHDB checker: http://www.swende.se/index.php/2010/04/googlyhacks-ghdb/ Subdomain-finder: http://www.swende.se/index.php/2010/06/finding-subdomains/ App finder (well, really the same as reverse-ip) : http://www.swende.se/index.php/2010/07/finding-applications/ These three would be very simple to move over to Lua, the problematic thing is that the latter two requires api keys (which are simple for an individual to obtain though). However, if the feature that reads options from a file could be used for this, so a user does not have to specify his key every time he does this, it would be pretty sweet to have. Regards, Martin ps. Shinnok, thanks for the post about Qt! Definitely going to check it out! _______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://seclists.org/nmap-dev/
Current thread:
- [NSE] Two new scripts: http-google-email and http-reverse-ip Shinnok (Jul 27)
- Re: [NSE] Two new scripts: http-google-email and http-reverse-ip Patrik Karlsson (Jul 28)
- Re: [NSE] Two new scripts: http-google-email and http-reverse-ip Martin Holst Swende (Jul 31)
- Re: [NSE] Two new scripts: http-google-email and http-reverse-ip Patrik Karlsson (Jul 28)