Re: massping -> ultra_scan prototype completed

[Fyodor <fyodor () insecure org>]

On Fri, Jul 13, 2007 at 12:53:15PM -0600, David Fifield wrote:
> I have finished a prototype of my proposed changes that implement
> massping using ultra_scan (see http://seclists.org/nmap-dev/2007/q3/0023.html)
> for some background). You can check it out with

Hi David.  It looks good so far!  Hopefully other people will take a
look and do some testing too.

I noticed that you have an nmap-massping-migration and also
nmap-massping-test branches under
svn://svn.insecure.org/nmap-exp/david/.  I'm assuming we should test
the -migration since that's the one you mentioned in your email.

> I'd like to ask for your feedback on how I've done the migration.
> r5095:5125 (svn diff -r 5095:5125) are small changes and bugfixes that
> shouldn't be controversial. r5126 (svn diff -c 5126) is the big change.

I think you probably mean revisions 5215 and 5216 rather than
5125/5126.  And "-r" instead of "-c".

I used Nmap -iR to create a list of 10,000 IPs as so:

nmap -n -iR 10000 -sL | egrep '^Host ' | awk '{print $2}' > ! pingtest.ips

Then I tried to run your latest code from nmap-massping-migration against those 10,000 IPs.  But I repeatedly got this 
error:

#./nmap -n -T4 -sP -PS80 -v -oA logz/newping-t1 -iL pingtest.ips

Starting Nmap 4.22SOC2 ( http://insecure.org ) at 2007-07-23 22:25 PDT
Initiating Ping Scan at 22:25
Scanning 2048 hosts [1 port/host]
Ping Scan Timing: About 7.45% done; ETC: 22:32 (0:06:16 remaining)
Your system does not support select()ing on pcap devices (Interrupted system call). PLEASE REPORT THIS ALONG WITH 
DETAILED SYSTEM INFORMATION TO THE nmap-dev MAILING LIST!
QUITTING!

This is probably a bug in the recent pcap-select patch rather than
your changes, but your changes just happen to expose the issue.  It
doesn't seem to happen with the old system.  The solution is probably
just to test for EINTR and then retry the select rather than quitting.
You can likely make this more likely to happen by pressing keys during
the scan to cause the signal interrupts.

> I kept the nexthost/massping interface. massping is now just a thin
> wrapper around ultra_scan:

Sounds like a good approach.

> As I proposed, the new ping scanning code reads its ping type and lists
> of ports directly from the global NmapOps data structure. I do not think
> changing the way this is done will be too difficult.

Do whatever you think is cleanest.  If it can easily be expressed as a
function argument, that seems better.  But if doing so requires a
bunch of harry code, then don't worry about it.

> scan_engine.cc is getting really huge, pushing 4500 lines (it was about
> 4000 before the migration). There are many common blocks of code that
> could be factored out, but I'd like to do that as a separate step. For
> long-term maintainability, the logic for the different scan types (ARP
> ping, ping, port, IP protocol) should be broken out into helper classes
> that are called by ultra_scan.

That sounds great to me.  And splitting that into a different step
absolutely makes sense.

> The new ping scanning code doesn't short-circuit; that is, it will keep
> sending probes even after a host's state is known.

Short circuiting is critical, so I was glad to see that checked into
your -test branch today!

> I don't have a feel for the timing aspects of the code. I don't feel
> qualified to tell if ultra_scan's timing is appropriate for ping scan or
> not. For example, there's a chunk in the old massping
>
>       /* Decide on the timeout, based on whether we need to also watch for TCP stuff */
>       if (ptech->icmpscan && !ptech->rawtcpscan && !ptech->rawudpscan) {
>         /* We only need to worry about pings, so we set timeout for the whole she-bang! */

This one shouldn't be needed for ultra_scan() since it already has
better algorithms which prevent the need for this little hack.

An important aspect of the migration project is testing and perhaps a
little tuning.  Trying various (unintrusive -- like port 80 pings)
host enumeration tests on, say, 50,000 IPs generated with -iR and
comparing the results (time taken and hosts discovered) with the old
system should be quite instructive!  The goals for this project are to
clean up the code, make host discovery faster, and make it more
accurate.  A tall order, I know :).

Cheers,
-F


_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org