Re: Slow name-resolution of very large target list

[Brandon Enright <bmenrigh () ucsd edu>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 22 May 2008 14:13:49 -0700
Fyodor <fyodor () insecure org> wrote:

> On Thu, May 22, 2008 at 07:16:31AM +0000, Brandon Enright wrote:
> > I've tried the scan from another network that has access to many
> > very fast local DNS servers and have specified them with
> > --dns-servers but that didn't seem to make any noticeable
> > difference.
> >
> > I tried adjusting these parameters in nmap_dns.cc:
> >
> > #define CAPACITY_MIN 10
> > #define CAPACITY_MAX 200
> > #define CAPACITY_UP_STEP 2
> >
> > but they didn't seem to have any noticeable effect either.
>
> I'm afraid these values are for Nmap's mass _reverse DNS_ subsystem.

D'oh!  That didn't even cross my mind.  This is the first time I've
given Nmap more than 1 or 2 host names.

> It is extremely common that Nmap needs to do massive rDNS because
> people scan huge networks (normally specifing the IP range) and Nmap
> by default does rDNS for every host which is found to be up.  Yet it
> is rare that Nmap has to do a lot of forward resolution.  When people
> specify DNS names, they usually only specify a small number.

Yep, the new system blazes for doing reverse-lookups.

> Due to this, Nmap only has a subsystem for parallel rDNS.  For forward
> DNS, Nmap just uses gethostbyname() in TargetGroup.cc.  I'm not sure
> if changing that is worthwhile, since it may cause more annoyance for
> people than it helps.  There are some advantages to gethostbyname(),
> since you are resolving in the same way as other applications in the
> system.  So various custom configurations are well supported, and we
> don't have to maintain or debug any of it.  Also, Nmap goes through
> target specifiers one at a time.  To do parallel forward DNS, Nmap
> would have to go through them all up front to figure out which ones
> were hostnames and resolve them in batches.

A reasonable argument for keeping it.

> So you may be best off using a mass DNS tool of some sort and passing
> those results to Nmap.  Or maybe you can configure your DNS system to
> time out queries more quickly?
>
> Cheers,
> -F

I'm working on cooking up a perl script to use the adns library.  I'm
not sure how portable adns is but perhaps we could either look into
using it or look into extending Doug's rDNS system to support
forward-lookups too.  We could have the --system-dns fall back on
gethostbyname().

Perhaps it is rare enough for someone to be doing what I'm doing that
it isn't worth the effort?

Thanks for the note, I was pulling my hair out yesterday tyring to
speed this scan up.

Brandon

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkg15FMACgkQqaGPzAsl94IB2gCgpEiJ9zwfHOsSBwvxsZhjMmVc
0VUAn1fOXpJBsDTCI6D9erC6efOzwyag
=SdxB
-----END PGP SIGNATURE-----

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org