Re: [NSE] Two new scripts: http-google-email and http-reverse-ip

[Patrik Karlsson <patrik () cqure net>]

On Jul 27, 2011, at 11:01 PM, Shinnok wrote:

> Hi nmap-dev,
>
> As stated in some of my previous e-mails, I started tinkering with NSE
> and LUA a bit, just so I don't miss this part of Nmap fun and I want to
> share with you two scripts I've written:
>
> 1. Google Web Search and Google Groups Search e-mail discovery
>
> This one attempts to search for e-mails pertaining to a specific domain
> in Google's Web search engine(google.com) and Google Groups search
> engine(groups.google.com).
>
> Sample output:
> ./nmap -p80 --script http-google-email insecure.org
>
> Starting Nmap 5.59BETA1 ( http://nmap.org ) at 2011-07-27 23:53 EEST
> Nmap scan report for insecure.org (74.207.254.18)
> Host is up (0.19s latency).
> rDNS record for 74.207.254.18: web.insecure.org
> PORT   STATE SERVICE
> 80/tcp open  http
> | http-google-email:
> | nmap-dev () insecure org
> | nmap-svn () insecure org
> |_fyodor () insecure org
>
> Nmap done: 1 IP address (1 host up) scanned in 4.01 seconds
>
>
>
> 2. Reverse domain ip lookup using Bing
>
> This one attempts to find domains that are hosted on a specific ip
> address using Bing's ip: operator.
>
> Sample output:
> ./nmap -p80 --script http-reverse-ip  insecure.org
>
> Starting Nmap 5.59BETA1 ( http://nmap.org ) at 2011-07-27 23:55 EEST
> Nmap scan report for insecure.org (74.207.254.18)
> Host is up (0.20s latency).
> rDNS record for 74.207.254.18: web.insecure.org
> PORT   STATE SERVICE
> 80/tcp open  http
> | http-reverse-ip:
> | nmap.org
> | insecure.org
> | secwiki.org
> |_images.insecure.org
>
> Nmap done: 1 IP address (1 host up) scanned in 1.75 seconds
>
> The both take a "pages" argument that specifies the number of results
> pages to request from search engines(the default is 5) and a domain and
> host argument respectively, that can be used to query a different target
> then the one nmap scans.
>
> I used http-google-malware.nse as a reference and the results are far
> from perfect and since they are my first attempts at NSE I am welcome to
> suggestions, improvements and conformance hints.
>
> Thanks,
> Shinnok
> <http-google-email.nse><http-reverse-ip.nse>_______________________________________________
> Sent through the nmap-dev mailing list
> http://cgi.insecure.org/mailman/listinfo/nmap-dev
> Archived at http://seclists.org/nmap-dev/


Hi Shinook,

First off, welcome to the great world of NSE :)
I've had some experience in the past with "wrapping" google search with tools such as wget.
What usually happens after a bunch of queries is that Google block the search possibility from that IP or redirect you 
to a captcha site or a site making heavy use of javascript to prevent further "automated searches".
In the past you were able to get a google API key to get around this problem and use a dedicated 3rd party API that 
wouldn't impose these restrictions.
I'm not sure how this works now. I also think Bing had some similar restrictions requiring an API key.

I know that Martin has done some work in this area and may have some newer information.

//Patrik
--
Patrik Karlsson
http://www.cqure.net
http://www.twitter.com/nevdull77

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/