Nmap Development mailing list archives
Re: ultra_scan-based host discovery now completed; testing needed
From: Fyodor <fyodor () insecure org>
Date: Tue, 31 Jul 2007 23:34:43 -0700
On Tue, Jul 31, 2007 at 08:39:26PM -0600, David Fifield wrote:
Testing and tuning are needed to make it work as well as the old massping. The new code is slower and detects fewer hosts than the old code.
Hmm. Maybe we did this the wrong way around. If massping is faster, then please start over and migrate ultra_scan to massping() instead. Just kidding :). I'm not sure why ultra_scan is slower, but I did do a lot of tuning on massping() so that may be a major factor. I'm pretty sure though that with tuning you can make the ultra_scan() ping faster than massping() while being at least as accurate. The good news is that optimizations to ultra_scan() will affect port scans too, so you may speed them up in the process. The bad news is that optimizations to ultra_scan() will affec\t port scans too, so if you're not careful you may screw them up in the process :). Be sure to take some benchmarks of doing port scans with the old system so that you can verify that you changes don't hurt performance or accuracy there. I'm thinking a big reason for the discrepancy might be that massping() may have used a much larger group size (number of hosts scanned in parallel). Look at the sizes which massping() used to use, and you may be able to increase ultra_scan to use similar sizes in those cases. I wouldn't make the sizes based on whether a ping scan is being done. Instead, I'd make it (roughly) proportional to the number of probes being done per host. If you're only doing 2 probes per host (not counting retransmissions), which is the ping scan default, you want to be scanning a very large number of hosts per parallel. But if you have thousands of probes per host, scanning (say) a thousand hosts in parallel will just waste a lot of RAM. You normally only do that many probes per host for a port scan, but one of the benefits of your new system is that it will be able to enable this for a ping scan, though we may need to change some structures before it will work. When you do this sort of tuning, --packet-trace is your friend. You saw a very dramatic difference in time with just 50 hosts scanned. So just run that scan with --packet-trace in both cases and study the difference. Given that the new system takes more than twice as long in this particular case, some of the most important reasons may be obvious once you look at the packet trace. Another important aspect is to use a diverse set of tests. Try some with only one probe per host (for example -sP -PE) and others with a dozen or two dozen probes per host. Try some against random IP lists, and others against consecutive IPs (e.g. www.whatever.com/24 or /16). Try some on your home network of 192.168.0.0/16 or 10.0.0.0/8. And even consider trying something like 127.0.0.0/20 (this may not work properly on all machines with a raw scan, but you could test it as a nonroot user which should use the connect() technique). Remebering to test the connect() behavior is important. Remember to pay attention to both accuracy and speed. In an ideal world, tuning would make your system faster and at least as accurate in every different situation you place it in. But that may not be realistic. Yet I have confidence that you can make your system work better in *most* cases. Feel free to call or IM me on Wednesday to talk strategy. I'm leaving for Defcon on Thursday so won't be very accessible there until I return home Sunday night. It must be a little frustrating to do all this work and find that it is slower and less accurate than before. But I believe that some good tuning will fix that, and maybe improve port scanning at the same time! So we will then have a faster, more accurate, and more powerful shared engine rather than two different ones which basically serve the same function. This is definitely one of this Summer's most exciting SoC projects. A small fraction of Nmap users will user some of the more advanced new functionality, but EVERYONE who uses Nmap will benefit from faster, more accurate host discovery and port scanning. Cheers, -F _______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://SecLists.Org
Current thread:
- ultra_scan-based host discovery now completed; testing needed David Fifield (Jul 31)
- Re: ultra_scan-based host discovery now completed; testing needed Kris Katterjohn (Jul 31)
- Re: ultra_scan-based host discovery now completed; testing needed Fyodor (Jul 31)