Re: [NSE] WHOIS

[jah <jah () zadkiel plus com>]

On 03/02/2008 22:47, Fyodor wrote:
> I'm wondering if people will get their IP addresses banned for doing
> too many whois queries?  It is very common that people scan
> consecutive ranges.  What do you think about caching the resulting
> net ranges?  So if someone scans 159.93.0.0/16, the first machine (159.93.0.0) would
> show:
>
> Host script results:
> |  WHOIS: Record found at whois.ripe.net
> |  inetnum: 159.93.0.0 - 159.93.255.255
> |  netname: JINR-NET
> |  descr: Joint Institute for Nuclear Research
> |_ country: RU
>
> But then for 159.93.0.1, the script would first look up its cached
> entries and see that it already has results which include that IP.
> Then it could either just include the same information or (probably
> better) an abbreviated entry like we do with traceroute results.  For
> example, maybe it could just give the netname or some other field, and
> a pointer to 159.93.0.0 for full results.
This is a great idea and one I'd not considered.  My testing of this 
script has been based around querying the largest number of different 
addresses in an attempt to cover the widest possible spectrum of responses.

On the face of it, the idea seems very easy to implement - and in terms 
of reducing the number of queries for a given range, it is trivial for 
most cases.
Where a record has been found for a target and where the record contains 
the net range details the script can store host.ip and the net range to 
nmap.registry.  Each instance of the script can check whether its target 
host.ip is within any of the ranges stored in the registry and return a 
pointer to the stored ip just as you suggested.  The script already has 
a function for testing whether an address is within a particular range - 
ip_in_net(ip, range) - so the script shouldn't need to grow very much 
bigger than its already monstrous size!

However...
By the time a record has been found for a target and the details stored 
to nmap.registry, several script instances (depending, I think, on 
timing) have already passed the point at which they check the registry 
and have begun their queries.  The number of such instances ranges 
between 3 and 8 in my testing thus far.
I really like the idea of pointing to an existing Host Script Result 
rather than repeating the same result for each target, but the above 
means that 3-8 targets will show the entire result before the pointers 
start to be displayed.
Additionally, if a hostgroup is of the order ~3-8, the results will be 
even more inconsistent, with some hostgroups perhaps completing before 
any of the results are cached.

I'm not happy with this inconsistency, but I wonder if we can live with 
it given that the main objective of reducing the number of queries is 
achieved.  Or perhaps we should dispense with the pointer idea and have 
each target display the entire result (which would be a shame).

Strangely enough, the first target to have it's result cached is not 
necessarily the first script instance run (probably due to the network, 
I'm not sure at this point) and a small amount of code was needed to 
ensure that only unique net ranges are stored in the registry and that 
each is associated with the lowest host.ip

I'm trying to think whether there might be a way to pause script 
execution until the first instance for a given hostgroup has completed 
(only if the remaining target addresses are consecutively numbered).

One more consideration:
If a record does not exist for a target (e.g. ARIN responds with "No 
Match for <host.ip>") the script would have to determine whether the 
remaining targets are likely to yield the same result (i.e. the range of 
addresses for which there exists no records which could be determined by 
querying in turn for /24, /16 and /8 networks until a record is found).  
As far as I can see in the documentation, a script instance knows only 
of one target which makes this impossible...

I've more testing to do with what I have so far and I'll post my 
modifications when I'm sure I've not broken anything.  In the meantime, 
I'd be grateful for any suggestions/insight.

regards,

jah

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org