Nmap 4.85BETA1 pre-release!

[Fyodor <fyodor () insecure org>]

Hi All.  It has been over four months since the 4.76 release, so this
new 4.85BETA1 is a whopper!  It includes Ncat, Ndiff, David's
performance enhancements, major OS detection improvements, Ron's SMB
script and module suite, and hundreds of other changes!  With such a
dramatic new release, there are probably some bugs, and I'm hoping you
folks can find them!  So I'm sending the URLs in this message, but I
don't plan to link them from the download page for a few days.  And
I'll be standing buy to rebuild if serious problems are found.
Anyway, here are the goods:

SOURCE TARBALLS:
http://nmap.org/dist/nmap-4.85BETA1.tar.bz2
http://nmap.org/dist/nmap-4.85BETA1.tgz

WINDOWS:
http://nmap.org/dist/nmap-4.85BETA1-setup.exe
http://nmap.org/dist/nmap-4.85BETA1-win32.zip

MAC OS X:
http://nmap.org/dist/nmap-4.85BETA1.dmg

RPMS (LINUX):
http://nmap.org/dist/nmap-4.85BETA1-1.src.rpm
http://nmap.org/dist/nmap-4.85BETA1-1.i386.rpm
http://nmap.org/dist/ncat-4.85BETA1-1.i386.rpm
http://nmap.org/dist/nmap-4.85BETA1-1.x86_64.rpm
http://nmap.org/dist/ncat-4.85BETA1-1.x86_64.rpm
http://nmap.org/dist/zenmap-4.85BETA1-1.noarch.rpm

CRYPTO SIGS:
http://nmap.org/dist/sigs/

Please test it out and report any problems to nmap-dev!  And enjoy the
new release!  Here is the roughly 500-line changelog entry for this
release:

o Added Ncat, a much-improved reimplementation of the venerable Netcat
  tool which adds modern features and makes use of Nmap's efficient
  networking libraries.  Features include SSL support, proxy
  connections (client or server, socks4 or connect-based, with or
  without authentication, optionally chained), TCP and UDP connection
  redirection, connection brokering (facilitating connections between
  machines which are behind NAT gateways), and much more.  It is
  cross-platform (Linux, Windows, Mac, etc.) and supports IPv6 as well
  as standard IPv4.  See http://nmap.org/ncat/ for details.  It is now
  included in our binary packages (Windows, Linux, and Mac OS X), and
  built by default.  You can skip it with the --without-ncat configure
  option.  Thanks to Kris and David for their great work on this!

o Added the Ndiff utility, which compares the results of two Nmap
  scans and describes the new/removed hosts, newly open/cosed ports,
  changed operating systems, etc.  This makes it trivial to scan your
  networks on a regular basis and create a report (XML or text format)
  on all the changes.  See http://nmap.org/ndiff/ and ndiff/README for
  more information. Ndiff is included in our binary packages and built
  by default, though you can prevent it from being built by specifying
  the --without-ndiff configure flag.  Thanks to David and Michael
  Pattrick for their great work on this.

o Released Nmap Network Scanning: The Official Nmap Project Guide to
  Network Discovery and Security Scanning.  From explaining port
  scanning basics for novices to detailing low-level packet crafting
  methods used by advanced hackers, this book suits all levels of
  security and networking professionals. A 42-page reference guide
  documents every Nmap feature and option, while the rest of the book
  demonstrates how to apply those features to quickly solve real-world
  tasks.  It was briefly the #1 selling computer book on Amazon.
  Translations to the German, Korean, and Brazilian Portuguese
  languages are forthcoming.  More than half of the book is already
  free online.  For more, see http://nmap.org/book/.

o David spent more than a month working on algorithms to improve port
  scan performance while retaining or improving accuracy.  The changes
  are described at http://seclists.org/nmap-dev/2009/q1/0054.html. He
  was able to reduce our "benchmark scan time" (which involves many
  different scan types from many source networks to many targets) from
  1879 seconds to 1321 without harming accuracy.  That is a 30% time
  reduction!

o Introduced the NSE documentation portal, which documents every NSE
  script and library included with Nmap. See http://nmap.org/nsedoc/.
  Script documentation was improved substantially in the process.
  Scripts and libraries must use the new NSEDoc format, which is
  described at http://nmap.org/book/nsedoc.html.  Thanks to Patrick
  and David for their great work on this.

o The 2nd Generation OS Detection System was dramatically improved for
  improved accuracy.  After substantial testing, David and Fyodor made
  the following changes:
  o The "T" (TTL test) result ranges were widened to prevent minor
    routing (and device hardware inconsistency) variations from causing
    so many matches to fail.
  o The TG (TTL guess) results were canonicalized. Nmap is only
    capable of assigning the values 0x20, 0x40, 0x80, and 0xFF for
    these tests, yet many fingerprints had different values.  This was
    due to bugs in our fingerprint integration tools.
  o The U1.TOS and IE.TOSI tests (both having to do with the IP Type
    of Service field) have been effectively eliminated (MatchPoints
    set to 0).  These proved particularly susceptible to false results
    due to networking hardware along the packet route manipulating the
    TOS header field.
  o An important bug in OS detection's congestion control algorithms
    was fixed.  It could lead to Nmap sending packets much too quickly
    in some cases, which hurt accuracy.

o Integrated all of your OS detection fingerprint submissions and
  corrections up to January 8.  The DB has grown more than 17% to
  1,761 fingerprints.  Newly detected services include Mac OS X
  10.5.6, Linux 2.6.28, iPhone 2.1, and all manner of WAPs, VoIP
  phones, routers, oscilloscopes, employee timeclocks, etc. Keep those
  submissions coming!

o Ron Bowes embarked on a massive MSRPC/NETBIOS project to allow Nmap
  to interrogate Windows machines much more completely.  He added
  three new nselib modules: msrpc, netbios, and smb. As the names
  suggest, they contain common code for scripts using MSRPC, NetBIOS,
  and SMB. These modules allow scripts to extract a great deal of
  information from hosts running Windows, particularly Windows
  2000. New or updated scripts using the modules are:
    nbstat.nse: get NetBIOS names and MAC address.
    smb-enum-domains.nse: enumerate domains and policies.
    smb-enum-processes.nse: allows a user with administrator
       credentials to view a tree of the processes running on the
       remote system (uses HKEY_PERFORMANCE_DATA hive).
    smb-enum-sessions.nse: enumerate logins and SMB sessions.
    smb-enum-shares.nse: enumerate network shares.
    smb-enum-users.nse: enumerate users and information about them.
    smb-os-discovery.nse: get operating system over SMB (replaces
      netbios-smb-os-discovery.nse).
    smb-security-mode.nse: determine if a host uses user-level or
      share-level security, and what other security features it
      supports.
    smb-server-stats.nse: grab statistics such as network traffic
      counts.
    smb-system-info.nse: get lots of information from the registry.

o A problem that caused OS detection to fail for most hosts in a
  certain case was fixed. It happened when sending raw Ethernet frames
  (by default on Windows or on other platforms with --send-eth) to
  hosts on a switched LAN. The destination MAC address was wrong for
  most targets. The symptom was that only one out of each scan group
  of 20 or 30 hosts would have a meaningful OS fingerprint. Thanks go
  to Michael Head for running tests and especially Trent Snyder for
  testing and finding the cause of the problem. [David]

o Zenmap now runs ndiff to for its "Compare Results" function. This
  completely replaces the old diff view. The diff window size is now
  more flexible for user resizing as well. [David]

o Added a Russian translation of the Nmap Reference Guide by Guz
  Alexander. We now have translations in 15 languages available from
  http://nmap.org/docs.html. More volunteer translators are welcome,
  as we are still missing some important languages. Translation
  instructions are available from that docs.html page.

o Update Windows installer to handle Windows 7 (tested with the Beta
  build 7000) [Rob Nicholls]

o Improved port scan performance by changing the list of high priority
  ports which Nmap shifts closer to the beginning of scans because
  they are more likely to be responsive.  We based the change on
  empirical data from large-scale scanning.  The new port list is:
    21, 22, 23, 25, 53, 80, 110, 111, 113, 135, 139, 143, 199, 256,
    443, 445, 554, 587, 993, 995, 1025, 1720, 1723, 3306, 3389, 5900, 
    8080, 8888 [Fyodor, David]

o [NSE] Almost all scripts were renamed to be more consistent.  They
  are now all lowercase and most of them start with the name of the
  service name they query.  Words are separated by hyphens. [David,
  Fyodor]

o [NSE] Now that scripts are better named, the "Id" field has been
  removed and the script name (sans the .nse or directory path
  information) is used in script output instead. [David]

o [NSE] Added banner.nse, a simple script which connects to open TCP
  ports and prints out anything sent in the first five seconds by the
  listening service. [Jah]

o [NSE] Added a new OpenSSL library with functions for multiprecision
  integer arithmetic, hashing, HMAC, symmetric encryption and
  symmetric decryption. [Sven]

o [Zenmap] Internationalization has been fixed [David]. Currently
  Zenmap has two translations:
    o German by Chris Leick
    o Brazilian Portuguese by Adriano Monteiro Marques (partial)
  For details on using an existing translation or localizing Zenmap
  into your own native language, see
  http://nmap.org/book/zenmap-lang.html. [David]

o Zenmap no longer outputs XML elements and attributes that are not in
  the Nmap XML DTD. This was done mostly by removing things from
  Zenmap's output, and adding a few new optional things to the Nmap
  DTD. A scan's profile name, host comments, and interactive text
  output are what were added to nmap.dtd. The .usr filename extension
  for saved Zenmap files is deprecated in favor of the .xml extension
  commonly used with Nmap. Because of these changes the
  xmloutputversion has been increased to 1.03. [David]

o The NSE registry now persists across host groups so that values
  stored in it will remain until they are explicitly removed or Nmap
  execution ends. [David]

o Enhanced the AS Numbers script (ASN.nse) to better consolidate
  results and bail out if the DNS server doesn't support the ASN
  queries. [Jah]

o Complete re-write of the marshaling logic for Microsoft RPC calls. 
  [Ron Bowes]

o Added a script that checks for ms08-067-vulnerable hosts
  (smb-check-vulns.nse) using the smb nselib. It also checks for an
  unfixed denial of service vulnerability Ron discovered in the
  Windows 2000 registry service. [Ron Bowes]

o [Zenmap] Text size is larger on Mac OS X thanks to a new included
  gtkrc file. [David]

o Reduced memory consumption for some longer-running scans by removing
  completed hosts from the lists after two minutes.  These hosts are
  kept around in case there is a late response, but this draws the
  line on how long we wait and hence keep this information in memory.
  See http://seclists.org/nmap-dev/2008/q3/0902.html for more. [Kris]

o The Windows installer now uses Zenmap binaries built using Python
  2.6.1 rather than 2.5.1 [Fyodor]

o When a system route can't be matched up directly with an interface
  by comparing addresses, Nmap now tries to match the route through
  another route. This helps for instance with a PPP connection where
  the default route's gateway address is routed through a different
  route, the one associated with the address of the PPP device. The
  problem would show itself as an inability to scan through the
  default route and the error message
    WARNING: Unable to find appropriate interface for system route to ...
  [David]

o Removed a code comment which simply declared /* WANKER ALERT! */ for
  no good reason. [Fyodor]

o NSE prints messages in debugging mode whenever a script starts or
  finishes [Patrick, David].

o [Ncat] The -l option can now be specified w/o a port number to
  listen on Ncat's default port number (31337).

o [Zenmap] The Nmap output window now scrolls automatically as a scan
  progresses. [David]

o [NSE] We now have a canonical way for scripts to check for
  dependency libraries such as OpenSSL.  This allows them to handle
  the issue gracefully (by exiting or doing some of their work if
  possible) rather than flooding the console with error messages as
  before. See http://nmap.org/nsedoc/modules/openssl.html. [Pattrick,
  David, Fyodor]

o Nmap now reports a proper error message when you combine an IPv6
  scan (-6) with random IPv4 address selection (-iR). [Henri Doreau]

o Nmap now builds with the _FORTIFY_SOURCE=2 define.  With modern
  versions of GCC, this adds extra buffer overflow protection and
  other security checks.  It is described at
  http://gcc.gnu.org/ml/gcc-patches/2004-09/msg02055.html. [David,
  Doug]

o The --excludefile option correctly handles files with no terminating
  newline instead of claiming "Exclude file line 0 was too long to
  read." [Henri Doreau]

o [NSE] Changed the datafiles library to remove constraining input
  checks, move nmap.fetch_file() to read_from_file(), and make
  get_array() and get_assoc_array() into normal functions. [Sven]

o [NSE] Fixed some bugs and typos in the datafiles library. [Jah]

o Nsock handles a certain Windows connect error, WSAEADDRNOTAVAIL
  (errno 10049), preventing an assertion failure that looked like
    Strange connect error from 203.65.42.255 (10049): No such file or directory
    Assertion failed: 0, file .\src\nsock_core.c, line 290
  The error could be seen by running a version scan against a
  broadcast address. Thanks to Tilo Köppe and James Liu for reporting
  the problem. [David]

o An "elapsed" attribute has been added to the XML output (in the
  "finished" tag), representing the total Nmap scanning time in
  seconds (floating point). [Kris]

o Fixed a division by zero error in the packet rate measuring code
  that could cause a display of infinity packets per seconds near the
  start of a scan. [Jah]

o Substantially updated the Nmap Scripting Engine guide/chapter
  (http://nmap.org/book/nse.html) so that it is up-to-date with all
  the latest NSE improvements.

o Fixed a bug in the IP validation code which would have let a specially
  crafted reply sent from a host on the same LAN slip through and cause
  Nmap to segfault.  Thanks to ithilgore of sock-raw.homeunix.org for
  the very detailed bug report. [Kris]

o [Zenmap] The crash reporter further enhances user privacy by showing
  all the information that will be submitted so you can edit it to
  remove identifying information such as the name of your home
  directory. If you provide an email address the report will be marked
  private so it will not appear on the public bug tracker. [David]

o [Zenmap] Zenmap now parses and records XSL stylesheet information
  from Nmap XML files, so files saved by Zenmap will be viewable in a
  web browser just like those produced by Nmap. [David]

o A possible Lua stack overflow in the DNS module was fixed. Lua detects
  these sorts of overflows and quits. [David]

o [NSE] Improved html-title script to support http-alt and https-alt
  (with SSL) and to handle a wider variety of redirects. [Jah]

o NSE scripts that require a list of DNS servers (currently only
  ASN.nse) now work when IPv6 scanning. Previously it gave an error
  message: "Failed to send dns query.  Response from dns.query(): 9".
  [Jah, David]

o [Zenmap] Added a workaround for a crash
    GtkWarning: could not open display
  on Mac OS X 10.5. The problem is caused by setting the DISPLAY
  environment variable in one of your shell startup files; that
  shouldn't be done under 10.5 and removing it will make other
  X11-using applications work better. Zenmap will now handle the
  situation automatically. [David]

o http-auth.nse now properly checks for default authentication
  credentials. A bug prevented it from working before. [Vlatko
  Kosturjak]

o Renamed irc-zombie.nse to auth-spoof and improved its description
  and output a bit. [Fyodor]

o Removed some unnecessary "demo" category NSE scripts: echoTest,
  chargenTest, showHTTPVersion, and showSMTPVersion.nse.  Moved
  daytimeTest from the "demo" category to "discovery".  Removed
  showHTMLTitle from the "demo" category, but it remains in the
  "default" and "safe" categories. This leaves just smtp-open-relay in
  the undocumented "demo" category. [Fyodor]

o [NSE] Removed ripeQuery.nse because we now have the much more robust
  whois.nse which handles all the major registries. [Fyodor]

o [NSE] Removed showSSHVersion.nse. Its only real claim to fame was
  the ability to trick some SSH servers (including at least OpenSSH
  4.3p2-9etch3) into not logging the connection.  This trick doesn't
  seem to work with newer versions of OpenSSH, as my
  openssh-server-4.7p1-4.fc8 does log the connection. Without the
  stealth advantage, the script has no real benefit over version
  detection or the upcoming banner grabbing script. [Fyodor]

o [Zenmap] Profile updates: The -sS option was added to the "Intense
  scan plus UDP" and "Slow comprehensive scan" profiles.  The -PN (ping
  only) option was added to "Quick traceroute". [David]

o [NSE} The smtp-commands script output is now more compact. [Jason
  DePriest, David]

o [Zenmap] Added a simple workaround for a bug in PyXML (an add-on
  Python XML library) that caused a crash. The crash would happen when
  loading an XML file and looked like "KeyError: 0". [David]

o A crash caused by an incorrect test condition was fixed. It would
  happen when running a ping scan other than a protocol ping, without
  debugging enabled, if an ICMP packet was received referring to a
  packet that was not TCP, UDP, or ICMP. Thanks to Brandon Enright and
  Matt Castelein for reporting the problem. [David]

o [Zenmap] The keyboard shortcut for "Save to Directory" has been
  changed from Ctrl+v to Ctrl+Alt+s so as not to conflict with the
  usual paste shortcut [Jah, Michael].

o Nmap now quits if you give a "backwards" port or protocol range like
  -p 20-10. The issue was noted by Arturo "Buanzo" Busleiman. [David]

o Fixed a bug which caused Nmap to infer an improper distance against
  some hosts when performing OS detection against a group whose
  distance varies between members. [David, Fyodor]

o [Zenmap] Host information windows are now like any other windows,
  and will not become unclosable by having their controls offscreen.
  Thanks to Robert Mead for the bug report.

o [NSE] showHTMLTitle can now follow (non-standard) relative
  redirects, and may do a DNS lookup to find if the redirected-to host
  has the same IP address as the scanned host. [Jah]

o [NSE] Enhanced the tohex() function in the stdnse library to support
  strings and added options to control the formatting. [Sven]

o [NSE] The http module tries to deal with non-standards-compliant
  HTTP traffic, particularly responses in which the header fields are
  separated by plain LF rather than CRLF. [Jah, Sven]

o [Zenmap] The help function now properly converts the pathname of the
  local help file to a URL, for better compatibility with different
  web browsers. [David]
  This should fix the crash
  WindowsError: [Error 2] The system cannot find the file specified:
  'file://C:\\Program Files\\Nmap\\zenmap\\share\\zenmap\\docs\\help.html'

o [NSE] Fixed a number of small bugs in the Nmap library
  (nse_nmaplib.cc), as described at
  http://seclists.org/nmap-dev/2008/q4/0663.html [Patrick]

o The HTTP_open_proxy.nse script was updated to match Google Web
  Server's changed header field: "Server: gws" instead of
  "Server: GWS/".  [Vlatko Kosturjak]

o Enhanced the ssh service detection signatures to properly
  detect protocol version 2 services. [Matt Selsky]

o Nsock now uses fselect() to work around problems with select() not
  working properly on non-socket descriptors on Windows.  This was
  needed for Ncat to work properly on that platform. See
  http://seclists.org/nmap-dev/2008/q3/0766.html. [Kris]

o Removed trailing null bytes from Ncat's responses in HTTP proxy
  mode. [David]

o [NSE] daytime.nse now runs against TCP ports in addition to the UDP
  ports it already handled. The output format was also
  improved. [David]

o XML output now contains the full path to nmap.xml on Windows. The
  path is converted to a file:// URL to provide better compatibility
  across browsers. [Jah]

o Made DNS timeouts in NSE a bit more aggressive at higher timing
  levels such as -T4 and -T5. [Jah]

o A script could be executed twice if it was given with the --script
  option, also in the "version" category, and version detection (-sV)
  was requested. This has been fixed. [David]

o Fixed port number representation in some Nmap and Nsock message
  output.  Incorrect conversion modifiers caused high ports to wrap
  around and be shown as negative values. [Kris]

o Upgraded the shipped libdnet library to version 1.12 (with our
  modifications). [Kris]

o Upgraded the OpenSSL binaries shipped in our Windows installer to
  version 0.9.8i. [Kris]

o [NSE] The SSLv2-support script no longer prints duplicate cyphers if
  they exist in the server's supported cypher list. [Kris]

o Fix compilation w/IPv6 support on Solaris by checking for inet_addr
  in -lnsr before using APR_CHECK_WORKING_GETNAMEINFO in
  configure. [David]

o Removed the nbase_md5.* and nbase_sha1.* files because our
  new nse_openssl library includes that functionality. [David]

o The robots.txt NSE script is now silent when there are no
  interesting results, rather than printing that robots.txt "is empty
  or has no disallowed entries". [Kris]

o Fixed a file (socket) descriptor leak which could occur when connect
  scan probes receive certain unusual error messages (including
  EHOSTUNREACH, and EHOSTDOWN). This led to error messages such as
  "Socket creation in sendConnectScanProbe: Too many open files (24)"
  [David]

o [Zenmap] Made floating host details windows into normal toplevel
  windows. This avoid a problem where the edge of a window could be
  off the edge of a screen and it would not be closable. The bug was
  reported by Robert Mead. [David]

o Use TIMEVAL_AFTER(...) instead of TIMEVAL_SUBTRACT(...) > 0 when
  deciding whether a probe response counts as a drop for scan delay
  purposes.  This prevents an integer overflow which could
  substantially degrade scan performance. [David]

o Reorganized macosx/Makefile to make it easier to add in new packages
  such as Ncat and Ndiff. Also removed the bogus clean-nmap and
  clean-zenmap targets. [David]

o [Zenmap] Fixed a crash related to the use of NmapOptions in
  ScanNotebook.py using the old interface (ops.num_random_targes,
  ops.input_filename) rather than the newer dict-style
  interface. [Jah]

o Split parallel DNS resolution and system DNS resolution into
  separate functions. Previously system DNS resolution was encapsulated
  inside the parallel DNS function, inside a big if block. Now the if
  is on the outside and decides which of the two functions to
  call. [David]

o [NSE] Remove "\r\r" in script output. If you print "\r\n", the
  Windows C library will transform it to "\r\r\n". So we just print
  "\n" with no special case for Windows.  Also fixed
  showSMTPversion.nse so that it doesn't print "\r\r" in the first
  place. [David]

o Updated IANA assignment IP list for random IP (-iR)
  generation. [Kris]

o OS scan point matching code can now handle tests worth zero
  points. We now assign zero points to ignore a couple tests which
  proved ineffective. [David]

o [Zenmap] Catch the exceptions that are caused when there's no XML
  output file, an empty one, or one that's half-complete. You can
  cause these three situations, respectively, with: "nmap -V", "nmap
  --iflist", or "nmap 0".  Also remove the target requirement for scans
  because you should be able to run commands such as "nmap --iflist"
  from Zenmap. [David]

o [Zenmap] Guard against the topology graph becoming empty in the
  middle of an animation.  This could happen if you removed a scan
  from the list of scans during an animation. The error looked like:
    File "usr/lib/python2.5/site-packages/radialnet/gui/RadialNet.py",
    line 1533, in __livens_up AttributeError: 'NoneType' object has no
    attribute 'get_nodes' [David]

o [Zenmap] Fixed a crash which could occur when you entered a command
  containing only whitespace.  David fixed various other possible
  crashes found in the crash report tracker too.  Zenmap users really
  are capable of finding every possible edge case which could cause a
  crash :).


Enjoy!
-Fyodor


_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org