Nmap 4.75 released

[Fyodor <fyodor () insecure org>]

Hi Everyone.  I'm delighted to report the release of Nmap 4.75, which
has almost 100 significant improvements since 4.68.  Some which I'm
most excited about are:

o While Nmap stands for "Network Mapper", it hasn't been able to
  actually draw you a map of the network--until now!  Visit
  http://nmap.org/book/zenmap-topology.html for details and pretty
  pictures of Zenmap's new Scan Topology system.

o I spent much of this summer scanning tens of millions of IPs on the
  Internet (plus collecting data contributed by some enterprises) to
  determine the most commonly open ports.  Nmap now uses that
  empirical data to scan more effectively.

And there is much more, from hundreds of new OS detection fingerprints
to many new Nmap Scripting Engine scripts and libraries.  I had no
idea how many people still used Windows 2000 until 4.68 came out
broken on that platform and I was flooded with email!  That is fixed
now.  And its just one of many bug fixes and performance improvements
in this release.  Remember that we had 7 Google SoC students working
full-time this summer, and this release includes some of their best
work.

You can obtain Nmap 4.75 from the normal location:

http://nmap.org/download.html 

Please give it a try! And if you encounter any problems, report them
to nmap-dev as described at http://nmap.org/book/man-bugs.html

Here is the detailed list of important 4.75 changes from
http://nmap.org/changelog.html:

o [Zenmap] Added a new Scan Topology system. The idea is that if we
  are going to call Nmap the "Network Mapper", it should at least be
  able to draw you a map of the network!  And that is what this new
  system does. It was achieved by integrating the RadialNet Nmap
  visualization tool (http://www.dca.ufrn.br/~joaomedeiros/radialnet),
  into Zenmap. Joao Medeiros has been developing RadialNet for more
  than a year. For details, complete with some of the most beautiful
  Zenmap screen shots ever, visit
  http://nmap.org/book/zenmap-topology.html. The integration work was
  done by SoC student Vladimir Mitrovic and his mentor David Fifield.

o [Zenmap] Another exciting new Zenmap feature is Scan Aggregation.
  This allows you to visualize and analyze the results of multiple
  scans at once, as if they were from one Nmap execution. So you might
  scan one network, analyze the results a bit, then scan some of the
  machines more intensely or add a completely new subnet to the
  scan. The new results are seamlessly added to the old, as described
  at http://nmap.org/book/zenmap-scanning.html#aggregation. [David,
  Vladimir]

o Expanded nmap-services to include information on how frequently each
  port number is found open.  The results were generated by scanning
  tens of millions of IPs on the Internet this Summer, and augmented
  with internal network data contributed by some large
  organizations. [Fyodor]

o Nmap now scans the most common 1,000 ports by default in either
  protocol (UDP scan is still optional).  This is a decrease from
  1,715 TCP ports and 1,488 UDP ports in Nmap 4.68.  So Nmap is faster
  by default and, since the port selection is better thanks to the
  port frequency data, it often finds more open ports as
  well. [Fyodor]

o Nmap fast scan (-F) now scans the top 100 ports by default in either
  protocol.  This is a decrease from 1,276 (TCP) and 1,017 (UDP) in
  Nmap 4.68. Port scanning time with -F is generally an order of
  magnitude faster than before, making -F worthy of its "fast scan"
  moniker. [Fyodor]

o The --top-ports option lets you specify the number of ports you wish
  to scan in each protocol, and will pick the most popular ports for
  you based on the new frequency data.  For both TCP and UDP, the top
  10 ports gets you roughly half of the open ports.  The top 1,000
  (out of 65,536 possible) finds roughly 93% of the open TCP ports and
  more than 95% of the open UDP ports. [Fyodor, Doug Hoyte]

o David integrated all of your OS detection fingerprint and correction
  submissions from March 11 until mid-July.  In the process we reached
  the 1500-signature milestone for the 2nd generation OS detection
  system. We can now detect the newest iPhones, Linux 2.6.25, OS X
  Darwin 9.2.2, Windows Vista SP1, and even the Nintendo Wii. Nmap now
  has 1,503 signatures, vs. 1,320 in 4.68. Integration is now faster
  and more pleasant thanks to the new OSassist application developed
  by Nmap SoC student Michael Pattrick. See
  http://seclists.org/nmap-dev/2008/q3/0089.html and
  http://seclists.org/nmap-dev/2008/q3/0139.html for more details.

o Nmap now works with Windows 2000 again, after being broken by our
  IPv6 support improvements in version 4.65. A couple new dependencies
  are required to run on Win2K, as described at
  http://nmap.org/book/inst-windows.html#inst-win2k .

o [Zenmap] Added a context-sensitive help system to the Profile
  Editor.  You can now mouse-over options to learn more about what
  they are used for and their proper argument syntax. [Jurand Nogiec]

o When Nmap finds a probe during ping scan which elicits a response,
  it now saves that information for the port scan and later phases.
  It can then "ping" the host with that probe as necessary to collect
  timing information even if the host is not responding to the normal
  port scan packets. Previously, Nmap's port scan timing pings could
  only use information gathered during that port scan itself.  A
  number of other "port scan ping" system improvements were made at
  the same time to improve performance against firewalled hosts. For
  full details, see http://seclists.org/nmap-dev/2008/q3/0647.html
  [David, Michael, Fyodor]

o --traceroute now uses the timing ping probe saved from host
  discovery and port scanning instead of finding its own probe. The
  timing ping probe is always the best probe Nmap knows about for
  eliciting a response from a target. This will have the most effect
  on traceroute after a ping scan, where traceroute would sometimes
  pick an ineffective probe and traceroute would fail even though the
  target was up. [David]

o Added dns-safe-recursion-port and dns-safe-recursion-txid
  (non-default NSE scripts) which use the 3rd party dns-oarc.net
  lookup to test the source port and transaction ID randomness of
  discovered DNS servers (assuming they allow recursion at all).
  These scripts, which test for the "Kaminsky" DNS bugs, were
  contributed by Brandon Enright.

o Added whois.nse, which queries the Regional Internet Registries
  (RIRs) to determine who the target IP addresses are assigned
  to. [Jah]

o [Zenmap] Overhauled the default list of scan profiles based on
  nmap-dev discussion.  Users now have a much more diverse and useful
  set of default profile options. And if they don't like any of those
  canned scan commands, they can easily create their own in the
  Profile Editor! [David]

o Fyodor made a number of performance tweaks, such as:
  o increase host group sizes in many cases, so Nmap will now commonly
    scan 64 hosts at a time rather than 30
  o align host groups with common network boundaries, such as /24 or
   /25
  o Increase maximum per-target port-scan ping frequency to one every
    1.25 seconds rather than every five. Port scan pings happen
    against heavily firewalled hosts and the like when Nmap is not
    receiving enough responses to normal scan to properly calculate
    timing variables and detect packet drops.

o Added a new NSE binlib library, which offers bin.pack() and
  bin.unpack() functions for dealing with storing values in and
  extracting them from binary strings.  For details, see
  http://nmap.org/book/nse-library.html#nse-binlib . [Philip
  Pickering]

o Added a new NSE DNS library. See this thread:
  http://seclists.org/nmap-dev/2008/q3/0310.html [Philip Pickering]

o Added new NSE libraries for base64 encoding, SNMP, and POP3 mail
  operations.  They are described at
  http://seclists.org/nmap-dev/2008/q3/0233.html . [Philip Pickering]

o Added NSE scripts popcapa (retrieves POP3 server capabilities) and
  brutePOP3 (brute force POP3 authentication cracker) which make use
  of the new POP3 library. [Philip Pickering]

o Added the SNMPcommunitybrute NSE script, which is a brute force
  community string cracker. Also modified SNMPsysdescr to use the new
  SNMP library. [Philip Pickering]

o Fixed the SMTPcommands script so that it can't return multiple
  values (which was causing problems). Thanks to Jah for tracking down
  the problem and sending a fix for SMTPcommands. Then Patrick fixed
  NSE so it can handle misbehaving scripts like this without causing
  mysterious side effects.

o Added a new NSE Unpwdb (username/password database) library for
  easily obtaining usernames or passwords from a list.  The functions
  usernames() and passwords() return a closure which returns a new
  list entry with every call, or nil when the list is exhausted.  You
  can specify your own username and/or password lists via the script
  arguments userdb and passdb, respectively. [Kris]

o Nmap's Nsock-utilizing subsystems (DNS, NSE, version detection) have
  been updated to support the -S and --ip-options flags. [Kris]

o A new --max-rate option was added, which complements --min-rate. It
  allows you to specify the maximum byte rate that Nmap is allowed to
  send packets. [David]

o Added --ip-options support for the connect() scan (-sT). [Kris]

o Nsock now supports binding to a local address and setting IPv4
  options with nsi_set_localaddr() and nsi_set_ipoptions(),
  respectively. [Kris]

o Added IPProto Ping (-PO) support to Traceroute, and fixed support for
  IPProto Scan (-sO) and the ICMP Pings (-PE, -PP, -PM) in Traceroute
  as well.  These could cause Nmap to hang during Traceroute. [Kris]

o [Zenmap] Added a "Cancel" button for cancelling a scan in progress
  without losing any Nmap output obtained so far. [Jurand Nogiec]

o Improve the netbios-smb-os-discovery NSE script to improve target
  port selection and to also decode the system's timestamp from an SMB
  response. [Ron at SkullSecurity]

o Nmap now avoids collapsing large numbers of ports in open|filtered
  state (e.g. just printing that 500 ports are in that state rather
  than listing them individually) if verbosity or debugging levels are
  greater than two.  See this thread:
  http://seclists.org/nmap-dev/2008/q3/0312.html . [Fyodor]

o The NSE http library now supports chunked encoding. [Sven Klemm]

o The NSE datafiles library now has generic file parsing routines, and
  the parsing of the standard nmap data files (e.g. nmap-services,
  nmap-protocols, etc.) now uses those generic routines.  NSE scripts
  and libraries may find them useful for dealing with their own data
  files, such as password lists. [Jah]

o Passed the big revision 10,000 milestone in the Nmap project SVN
  server: http://seclists.org/nmap-dev/2008/q3/0682.html

o Added some Windows and MinGW compatibility patches submitted by
  Gisle Vanem.

o Improved nse_init so that compilation/runtime errors in NSE scripts
  no longer cause the script engine to abort. [Patrick]

o Fix a cosmetic bug in --script-trace hex dump output which resulting
  in bytes with the highest bit set being prefixed with ffffff. [Sven
  Klemm]

o Removed the nselib-bin directory. The last remaining shared NSE
  module, bit, has been made static by Patrick. Shared modules were
  broken for static builds of Nmap, such as those in the RPMS. We also
  had the compilation problems (particularly on OpenBSD) with shared
  modules which lead us to make PCRE static a while back. [David]

o Updated rpcinfo NSE script to use the new pack/unpack (binlib)
  functions, use the new tab library, include better documentation, and
  fix some bugs. [Sven Klemm]

o Add useful details to the error message printed when an NSE script
  fails to load (due to syntax error, etc.) [Patrick]

o Fix a bug in the NSE http library which would cause some scripts to
  give the error: SCRIPT ENGINE: C:\Program
  Files\Nmap\nselib/http.lua:77: attempt to call field 'parse' (a nil
  value) [Jah]

o Fixed a Makefile problem (race condition) which could lead to build
  failures when launching make in parallel mode (e.g. -j4). [Michal
  Januszewski]

o Added new addrow() function to NSE tab library.  It allows
  developers to add a whole row at once rather than doing a separate
  add() call for each column in a row. [Sven Klemm]

o Completion time estimates provided in verbose mode or when you hit a
  key during scanning are now more accurate thanks to algorithm
  improvements by David.

o Fixed a number of NSE scripts which used print_debug()
  incorrectly. See
  http://seclists.org/nmap-dev/2008/q3/0470.html. [Sven Klemm].

o [Zenmap] The Ports/Hosts view now provides full version detection
  values rather than just a simple summary. [Jurand Nogiec]
  
o [Zenmap] When you edit the command-entry field, then change the
  target selection, Nmap no longer blows away your edits in favor of
  using your current profile. [Jurand Nogiec]

o Nsock now returns data from UDP packets individually, preserving the
  packet boundary, rather than concatenating the data from multiple
  packets into a single buffer.  This fixes a problem related to our
  reverse-DNS system, which can only handle one DNS packet at a time.
  Thanks to Tim Adam of ManageSoft for debugging the problem and
  sending the patch.  Doug Hoyte helped with testing, and it was
  applied by Fyodor.

o [Zenmap] Fixed a crash which would occur when you try to compare two
  files, either of which has more than one extraports element. [David]

o Added the undocumented (except here) --nogcc option which disables
  global/group congestion control algorithms and so each member of a
  scan group of machines is treated separately.  This is just an
  experimental option for now. [Fyodor]

o [Zenmap] The Ports/Hosts display now has different colors for open
  and closed ports. [Vladimir]

o Fixed Zenmap so that it displays all Nmap errors.  Previously, only
  stdout was redirected into the window, and not stderr.  Now they are
  both redirected. [Vladimir]

o NSE can now be used in combination with ping scan (e.g. "-sP
  --script") so that you can execute host scripts without needing to
  perform a port scan. [Kris]

o [NSE] Category names are now case insensitive. [Patrick]

o [NSE] Each thread for a script now gets its own action closure (and
   upvalues). See: http://seclists.org/nmap-dev/2008/q2/0549.html
  [Patrick]

o [NSE] The script_scan_result structure has been changed to a class,
  ScriptResult, which now holds a Script's output in an std::string.
  This removes the need to use malloc and free to manage this memory.
  A similar change was made to the run_record structure. [Patrick]

o [NSE] Fixed a socket exhaustion deadlock which could prevent a
  script scan from ever finishing. Now, rather than limit the total
  number of sockets which can be open, we limit the number of scripts
  which can have sockets open at once.  And once a script has one
  socket opened, it is permitted to open as many more as it
  needs. [Patrick]

o A hashing library (code from OpenSSL) was added to NSE.  hashlib
  contains md5 and sha1 routines. [Philip Pickering]

o Fixed host discovery probe matching when looking at the returned TCP
  data in an ICMP error message.  This could formerly lead to
  incorrectly discarded responses and the debugging error message:
  "Bogus trynum or sequence number in ICMP error message" [Kris]

o Fixed a segmentation fault in Nsock which occurred when calling
  nsock_write() with a data length of -1 (which means the data is a
  NUL-terminated string and Nsock should take the length itself) and
  the Nsock trace level was at least 2. [Kris]

o The NSE Comm library now defaults to trying to read as many bytes as
  are available rather than lines if neither the "bytes" nor "lines"
  options are given.  Thanks to Brandon for reporting a problem which
  he noticed in the dns-test-open-recursion script. [Kris]

o Updated zoneTrans.nse to replace length bytes in returned domain
  names to periods itself rather than relying on NSE's old behavior of
  replacing non-printable characters with periods.  Thanks to Rob
  Nicholls for reporting the problem. [Kris]

o Some Zenmap crashes have been fixed: trying to "refresh" the output
  of a scan loaded from a file, and trying to re-save a file loaded
  from the command line in some circumstances. [David]

o [Zenmap] The file selector now remembers what directory it was last
  looking at. [David]

o Added an extra layer of validity checking to received packets
  (readip_pcap), just to be extra safe. See
  http://seclists.org/nmap-dev/2008/q3/0644.html . [Kris]

o Zenmap defaults to showing files matching both *.xml and *.usr in
  the file selector. Previously it only showed those matching *.usr.
  The new combined format will be XML and .usr will be deprecated.
  See http://seclists.org/nmap-dev/2008/q3/0093.html .

o Nmap avoids printing the sending rate in bytes per second during a
  TCP connect scan. Because the number of bytes per probe is not
  known, it used to print current sending rates: 11248.85 packets / s,
  0.00 bytes / s.  Now it will print simply print rates like "11248.85
  packets / s". [David]

o [Zenmap] Nmap's installation process now include .desktop files
  which install menu items for launching Zenmap as a privileged or
  non-privileged process on Linux. This will mainly effect people who
  install nmap and Zenmap directly from the source code. [Michael]

o Improved performance of IP protocol scan by fixing a bug related to
  timing calculations on ICMP probe responses.  See r8754 svn log for
  full details. [David]

o Nmap --reason output no longer falsely reports a localhost-response
  during -PN scans. See
  http://seclists.org/nmap-dev/2008/q3/0188.html. [Michael]

o [Zenmap] The higwidgets Python package has moved so it is now a
  subpackage of zenmapGUI. This avoids naming conflicts with Umit,
  which uses a slightly different version of higwidgets. [David]

o A bug that could cause some host discovery probes to be incorrectly
  interpreted as drops was fixed. This occurred only when the IP
  protocol ping (-PO) option was combined with other ping
  types. [David]

o A new scanflags attribute has been added to XML output, which lists
  all user specified --scanflags for the scan. nmap.dtd has been
  modified to account for this. [Michael]

o The loading of the nmap-services file has been made much
  faster--roughly 9 times faster in common cases.  This is important
  for the new (much larger) frequency augmented nmap-services
  file. [David]

o Added a script (ASN.nse) which uses Team Cymru's DNS interface to
  determine the routing AS numbers of scanned IP addresses.  They even
  set up a special domain just for Nmap queries.  The script is still
  experimental and non-default. [Jah, Michael]

o [Zenmap] Clicking "Cancel" in a file chooser in the diff interface
  no longer causes a crash. [David]

o The shtool build helper script has been updated to version 2.0.8. An
  older version of shutil caused installation to fail when the locale
  was set to et_EE. Thanks to Michal Januszewski for the bug
  report. [David]

o [Zenmap] Removed services.dmp and os_dmp.dmp and all the files that
  referred to them. They are not needed with the new search
  interface. Also removed an unused search progress bar.  And some
  broken fingerprint submission code.  Yay for de-bloating! [David]

o [Zenmap] Added "%F" to the Exec link in the new Zenmap desktop
  file. We expect (hope) that this will allow dragging and dropping
  XML files onto the icon. [David]

o [Zenmap] The -o[XGASN] options can now be specified, just as you can
  at the console. [Vladimir]

o [Zenmap] You can now shrink the scan window below its default
  size thanks to NmapOutputViewer code enhancements. [David]

o [Zenmap] Removed optional use of the Psyco Python optimizer since
  Zenmap is not the kind of CPU-bound application which benefits from
  Psyco.

o [Zenmap] You can now select more than one host in the "Ports /
  Hosts" view by control-clicking them in the column at left.

o [Zenmap] The profile editor now offers the --traceroute option.

o Zenmap now uses Unicode objects pervasively when dealing with Nmap
  text output, though the only internationalized text Nmap currently
  outputs is the user's time zone. [David]

o Unprintable characters in NSE script output (which really shouldn't
  happen anyway) are now printed like \xHH, where HH is the
  hexadecimal representation of the character. See
  http://seclists.org/nmap-dev/2008/q3/0180.html . [Patrick]

o Nmap sometimes sent packets with incorrect IP checksums,
  particularly when sending the UDP probes in OS detection. This has
  been fixed. Thanks to Gisle Vanem for reporting and investigating the
  bug. [David]

o Fixed the --without-liblua configure option so that it works
  again. [David]

o In the interest of forward compatibility, the xmloutputversion
  attribute in Nmap XML output is no longer constrained to be a
  certain string ("1.02"). The xmloutputversion should be taken as
  merely advisory by authors of parsers.

o Zenmap no longer leaves any temporary files lying around. [David]

o Nmap only prints an uptime guess in verbose mode now, because in
  some situations it can be very inaccurate. See the discussion at
  http://seclists.org/nmap-dev/2008/q3/0392.html. [David]

Enjoy the release!
-Fyodor
_______________________________________________
Sent through the nmap-hackers mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-hackers
Archived at http://seclists.org