Re: Zmap detecting more hosts than Nmap

[Fyodor <fyodor () nmap org>]

On Fri, Jan 3, 2014 at 12:35 PM, Jacek Wielemborek <d33tah () gmail com> wrote:
> On 30C3, I heard an interesting talk by J. Alex Halderman, the author of
> ZMap.
> In his presentation, he - among other things - compared ZMap to Nmap,
> pointing
> out that despite its stateless approach, his tool actually finds more hosts
> compared to Nmap in its "aggresive" mode. His explanation can be found
> here:

Hi Jacek.  Thanks for sending the video link.  He keeps calling it an
"aggressive" mode, but he's actually using Nmap's "-T Insane" (-T5) mode
instead. He mentions this at 30:50 in the video and more details are in
their paper.  Basically they chose Nmap command lines which are both
terribly slow for what they are doing and also terribly inaccurate, then
they brag about how much faster and more accurate their system is. Well,
duh.  They never contacted us or we would have pointed out Nmap's fixed
rate scanning capability, which we added more than five years ago and which
would have been perfect in this case (along with huge hostgroup size,
disabling retransmissions, setting a reasonable rtt timeout, etc.)  He has
a whole slide talking about how Nmap's accuracy suffered because of low
timeout values, but they are the ones who chose a timeout value so low that
we document it as "insane mode".  I don't think this was a malicious
attempt to rig their benchmarks, but they clearly didn't spend much time
choosing an optimal Nmap command line.  I meant to mail him about this way
back when I read the Zmap paper, but I didn't, so I can't really fault him
for repeating the same BS numbers.  And at least he talked about how
valuable he finds Nmap to be in general, and admits that his comparison is
"a little bit unfair".

But ignoring that issue, I think Zmap is great and their research results
are very interesting.  Especially their data on accuracy vs. scan rate and
number of retransmissions.  Even acknowledging that they are on a better
network than 99% of us, the results were surprising in a good way.
 Masscan, Unicornscan, and Scanrand are also great tools which help solve a
very similar problem to Zmap (large Internet surveys from high bandwidth
hosts where accuracy isn't as critical as speed).  We added Nmap's fixed
rate scanning capability (e.g. --min-rate) long ago to address this need,
but I'm sure we can improve it further.  Tools like Zmap/Masscan/etc. may
only be able to do a tiny fraction of what Nmap can, but they do it VERY
quickly!  Even if they had properly used Nmap's --min-rate option, would we
have been able to keep up with Zmap's 1.5 million packets per second?  I
doubt it.  And Masscan is apparently even faster.  But there is no reason
Nmap shouldn't be able to saturate a 1Gbps or maybe even 10Gbps line.  I
think it should be one of our big priorities this year.  Plus it gives me a
great excuse to rent a 10Gbps server, which I've kind of dreamed about for
while :).

Cheers,
Fyodor
_______________________________________________
Sent through the dev mailing list
http://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/