Nmap 5.61TEST4 released - 51 New Scripts, web spidering, vuln library, and more!

[Fyodor <fyodor () insecure org>]

Hello folks, and happy new year!  I'd like to start 2012 off
right--with a new version of Nmap.  So I'm happy to release 5.61TEST4.
The version number may not sound that different than the previous
5.61TEST2, but we've made many big improvements in the last three
months.  These include:

o Patrik's spidering library for recursively crawling web sites, and a
  bunch of scripts to make use of it.

o Djalal and Henri's vulnerability management library (vulns.lua) for providing
  consistant output and reporting of discovered vulns.

o An incredible 51 new scripts, bringing the total to 297 in this
  release!

Also, to improve the user experience, the Windows installer now
installs various browser toolbars, search engine redirectors, and
associated adware.  Not!  We'd never pull a Download.com
(http://insecure.org/news/download-com-fiasco.html), but it emphasizes
why you should download Nmap from the true source:

http://nmap.org/download.html

Here are the most significant changes since 5.61TEST2:

o [NSE] Added a new httpspider library which is used for recursively
  crawling web sites for information.  New scripts using this
  functionality include http-backup-finder, http-email-harvest,
  http-grep, http-open-redirect, and http-unsafe-output-escaping. See
  http://nmap.org/nsedoc/ or the list later in this file for details
  on these. [Patrik]

o Our Mac OS X packages are now x86-only (rather than universal),
  reducing the download size from 30 MB to about 17.  If you still
  need a PowerPC version (Apple stopped selling those machines in
  2006), you can use Nmap 5.51 or 5.61TEST2 from
  http://nmap.org/dist/?C=M&O=D.

o We set up a new SVN server for the Nmap codebase.  This one uses SSL
  for better security, WebDAV rather than svnserve for greater
  functionality, is hosted on a faster (virtual) machine, provides
  Nmap code history back to 1998 rather than 2005, and removes the
  need for the special "guest" username.  The new server is at
  https://svn.nmap.org.  More information:
  http://seclists.org/nmap-dev/2011/q4/504.

o [NSE] Added a vulnerability management library (vulns.lua) to store and to
  report discovered vulnerabilities.  Modified these scripts to use
  the new library:
  - ftp-libopie.nse
  - http-vuln-cve2011-3192.nse
  - ftp-vuln-cve2010-4221.nse
  - ftp-vsftpd-backdoor.nse
  - smtp-vuln-cve2011-1720.nse
  - smtp-vuln-cve2011-1764.nse
  - afp-path-vuln.nse
  [Djalal, Henri]

o [NSE] Added a new script force feature.  You can force scripts to
  run against target ports (even if the "wrong" service is detected)
  by placing a plus in front of the script name passed to --script.
  See
  http://nmap.org/book/nse-usage.html#nse-script-selection. [Martin
  Swende]

o [NSE] Added 51(!) NSE scripts, bringing the total up to 297.  They
  are all listed at http://nmap.org/nsedoc/, and the summaries are
  below (authors listed in brackets):

  + amqp-info gathers information (a list of all server properties)
    from an AMQP (advanced message queuing protocol)
    server. [Sebastian Dragomir]

  + bitcoin-getaddr queries a Bitcoin server for a list of known
    Bitcoin nodes. [Patrik Karlsson]

  + bitcoin-info extracts version and node information from a Bitcoin
    server [Patrik Karlsson]

  + bitcoinrpc-info obtains information from a Bitcoin server by
    calling <code>getinfo</code> on its JSON-RPC interface. [Toni
    Ruottu]

  + broadcast-pc-anywhere sends a special broadcast probe to discover
    PC-Anywhere hosts running on a LAN. [Patrik Karlsson]

  + broadcast-pc-duo discovers PC-DUO remote control hosts and
    gateways running on the LAN. [Patrik Karlsson]

  + broadcast-rip-discover discovers hosts and routing information
    from devices running RIPv2 on the LAN. It does so by sending a
    RIPv2 Request command and collects the responses from all devices
    responding to the request. [Patrik Karlsson]

  + broadcast-sybase-asa-discover discovers Sybase Anywhere servers on
    the LAN by sending broadcast discovery messages. [Patrik Karlsson]

  + broadcast-wake-on-lan wakes a remote system up from sleep by
    sending a Wake-On-Lan packet. [Patrik Karlsson]

  + broadcast-wpad-discover Retrieves a list of proxy servers on the
    LAN using the Web Proxy Autodiscovery Protocol (WPAD). [Patrik
    Karlsson]

  + dns-blacklist checks target IP addresses against multiple DNS
    anti-spam and open proxy blacklists and returns a list of services
    where the IP has been blacklisted. [Patrik Karlsson]

  + dns-zeustracker checks if the target IP range is part of a Zeus
    botnet by querying ZTDNS @ abuse.ch. [Mikael Keri]

  + ganglia-info retrieves system information (OS version, available
    memory, etc.) from a listening Ganglia Monitoring Daemon or
    Ganglia Meta Daemon. [Brendan Coles]

  + hadoop-datanode-info discovers information such as log directories
    from an Apache Hadoop DataNode HTTP status page. [John R. Bond]

  + hadoop-jobtracker-info retrieves information from an Apache Hadoop
    JobTracker HTTP status page. [John R. Bond]

  + hadoop-namenode-info retrieves information from an Apache Hadoop
    NameNode HTTP status page. [John R. Bond]

  + hadoop-secondary-namenode-info retrieves information from an
    Apache Hadoop secondary NameNode HTTP status page. [John R. Bond]

  + hadoop-tasktracker-info retrieves information from an Apache
    Hadoop TaskTracker HTTP status page. [John R. Bond]

  + hbase-master-info retrieves information from an Apache HBase
    (Hadoop database) master HTTP status page. [John R. Bond]

  + hbase-region-info retrieves information from an Apache HBase
    (Hadoop database) region server HTTP status page. [John R. Bond]

  + http-apache-negotiation checks if the target http server has
    mod_negotiation enabled.  This feature can be leveraged to find
    hidden resources and spider a web site using fewer requests. [Hani
    Benhabiles]

  + http-backup-finder Spiders a website and attempts to identify
    backup copies of discovered files.  It does so by requesting a
    number of different combinations of the filename (e.g. index.bak,
    index.html~, copy of index.html). [Patrik Karlsson]

  + http-cors tests an http server for Cross-Origin Resource Sharing
    (CORS), a way for domains to explicitly opt in to having certain
    methods invoked by another domain. [Toni Ruottu]

  + http-email-harvest spiders a web site and collects e-mail
    addresses. [Patrik Karlsson]

  + http-grep spiders a website and attempts to match all pages and
    urls against a given string. Matches are counted and grouped per
    url under which they were discovered. [Patrik Karlsson]

  + http-method-tamper tests whether a JBoss target is vulnerable to
    jmx console authentication bypass (CVE-2010-0738). [Hani
    Benhabiles]

  + http-open-redirect spiders a website and attempts to identify open
    redirects. Open redirects are handlers which commonly take a URL
    as a parameter and responds with a http redirect (3XX) to the
    target. [Martin Holst Swende]

  + http-put uploads a local file to a remote web server using the
    HTTP PUT method. You must specify the filename and URL path with
    NSE arguments. [Patrik Karlsson]

  + http-robtex-reverse-ip Obtains up to 100 forward DNS names for a
    target IP address by querying the Robtex service
    (http://www.robtex.com/ip/). [riemann]

  + http-unsafe-output-escaping spiders a website and attempts to
    identify output escaping problems where content is reflected back
    to the user. [Martin Holst Swende]

  + http-vuln-cve2011-3368 tests for the CVE-2011-3368 (Reverse Proxy
    Bypass) vulnerability in Apache HTTP server's reverse proxy
    mode. [Ange Gutek, Patrik Karlsson"]

  + ipv6-node-info obtains hostnames, IPv4 and IPv6 addresses through
    IPv6 Node Information Queries. [David Fifield]

  + irc-botnet-channels checks an IRC server for channels that are
    commonly used by malicious botnets. [David Fifield, Ange Gutek]

  + irc-brute performs brute force password auditing against IRC
    (Internet Relay Chat) servers. [Patrik Karlsson]

  + krb5-enum-users discovers valid usernames by brute force querying
    likely usernames against a Kerberos service. [Patrik Karlsson]

  + maxdb-info retrieves version and database information from a SAP
    Max DB database. [Patrik Karlsson]

  + metasploit-xmlrpc-brute performs brute force password auditing
    against a Metasploit RPC server using the XMLRPC protocol. [Vlatko
    Kosturjak]

  + ms-sql-dump-hashes Dumps the password hashes from an MS-SQL server
    in a format suitable for cracking by tools such as
    John-the-ripper. In order to do so the user needs to have the
    appropriate DB privileges. [Patrik Karlsson]

  + nessus-brute performs brute force password auditing against a
    Nessus vulnerability scanning daemon using the NTP 1.2
    protocol. [Patrik Karlsson]

  + nexpose-brute performs brute force password auditing against a
    Nexpose vulnerability scanner using the API 1.1. [Vlatko
    Kosturjak]

  + openlookup-info parses and displays the banner information of an
    OpenLookup (network key-value store) server. [Toni Ruottu]

  + openvas-otp-brute performs brute force password auditing against a
    OpenVAS vulnerability scanner daemon using the OTP 1.0
    protocol. [Vlatko Kosturjak]

  + reverse-index creates a reverse index at the end of scan output
    showing which hosts run a particular service. [Patrik Karlsson]

  + rexec-brute performs brute force password auditing against the
    classic UNIX rexec (remote exec) service. [Patrik Karlsson]

  + rlogin-brute performs brute force password auditing against the
    classic UNIX rlogin (remote login) service. [Patrik Karlsson]

  + rtsp-methods determines which methods are supported by the RTSP
    (real time streaming protocol) server. [Patrik Karlsson]

  + rtsp-url-brute attempts to enumerate RTSP media URLS by testing
    for common paths on devices such as surveillance IP
    cameras. [Patrik Karlsson]

  + telnet-encryption determines whether the encryption option is
    supported on a remote telnet server.  Some systems (including
    FreeBSD and the krb5 telnetd available in many Linux
    distributions) implement this option incorrectly, leading to a
    remote root vulnerability. [Patrik Karlsson, David Fifield,
    Fyodor]

  + tftp-enum enumerates TFTP (trivial file transfer protocol) filenames by testing
    for a list of common ones. [Alexander Rudakov]

  + unusual-port compares the detected service on a port against the
    expected service for that port number (e.g. ssh on 22, http on 80)
    and reports deviations. [Patrik Karlsson]

  + vuze-dht-info retrieves some basic information, including protocol
    version from a Vuze filesharing node. [Patrik Karlsson]

o [NSE] Added some new protocol libraries
 + amqp (advanced message queuing protocol) [Sebastian Dragomir]
 + bitcoin crypto currency [Patrik Karlsson
 + dnsbl for DNS-based blacklists [Patrik Karlsson
 + rtsp (real time streaming protocol) [Patrik Karlsson]
 + httpspider and vulns have separate entries in this CHANGELOG

o Nmap now includes a nmap-update program for obtaining the latest
  updates (new scripts, OS fingerprints, etc.)  The system is
  currently only available to a few developers for testing, but we
  hope to enable a larger set of beta testers soon. [David]

o On Windows, the directory <HOME>\AppData\Roaming\nmap is now
  searched for data files. This is the equivalent of $HOME/.nmap on
  POSIX. [David]

o Improved OS detection performance by scaling congestion control
  increments by the response rate during OS scan, just as was done
  for port scan before. [David]

o [NSE] The targets-ipv6-multicast-*.nse scripts now scan all
  interfaces by default. They show the MAC address and interface name
  now too. [David, Daniel Miller]

o Added some new version detection probes:
 + MongoDB service [Martin Holst Swende]
 + Metasploit XMLRPC service [Vlatko Kosturjak]
 + Vuze filesharing system [Patrik]
 + Redis key-value store [Patrik]
 + memcached [Patrik]
 + Sybase SQL Anywhere [Patrik]
 + VMware ESX Server [Aleksey Tyurin]
 + TCP Kerberos [Patrik]
 + PC-Duo [Patrik]
 + PC Anywhere [Patrik]

o Targets requiring different source addresses now go into different
  hostgroups, not only for host discovery but also for port scanning.
  Before, only responses to one of the source addresses would be
  processed, and the others would be ignored. [David]

o Tidied up the version detection DB (nmap-service-probes) with a new
  cleanup/canonicalization program sv-tidy.  In particular, this:
 - Removes excess whitespace
 - Sorts templates in the order m p v i d o h cpe:
 - Canonicalizes template delimiters in the order: / | % = @ #.
 [David]

o The --exclude and --excludefile options for excluding targets can
  now be used together. [David]

o [NSE] Added support for detecting whether a http connection was established
  using SSL or not to the http.lua library [Patrik]

o [NSE] Added local port to BPF filter in snmp-brute to fix bug that would
  prevent multiple scripts from receiving the correct responses. The bug was
  discovered by Brendan Bird. [Patrik]

o [NSE] Changed the dhcp-discover script to use the DHCPINFORM request
  to query dhcp servers instead of DHCPDISCOVER. Also removed DoS code
  from dhcp-discover and placed the script into the discovery and safe
  categories. Added support for adding options to DHCP requests and
  cleaned up some code in the dhcp library. [Patrik]

o [NSE] Applied patch to snmp-brute that solves problems with handling
  errors that occur during community list file parsing. [Duarte
  Silva]

o [NSE] Added new fingerprints to http-enum for:
  - Subversion, CVS and Apache Archiva [Duarte Silva]
  - DVCS systems Git, Mercurial and Bazaar [Hani Benhabiles].

o [NSE] Applied some code cleanup to the snmp library. [Brendan Byrd]

o [NSE] Fixed an undeclared variable bug in snmp-ios-config [Patrik]

o [NSE] Add additional version information to Mongodb scripts [Martin
  Swende]

o [NSE] Added path argument to the http-auth script and update the
  script to use stdnse.format_output. [Duarte Silva, Patrik]

o [NSE] Fixed bug in the http library that would fail to parse
  authentication headers if no parameters were present. [Patrik]

o Made a syntax change in the zenmap.desktop file for compliance with
  the XDG standard. [Frederik Schwarzer]

o [NSE] Replaced a number of GET requests to HEAD in http-
  fingerprints.lua.  HEAD is quicker and sufficient when no matching
  is performed on the returned contents.  [Hani Benhabiles]

o [NSE] Added support for retrieving SSL certificates from FTP
  servers. [Matt Selsky]

o [Nping] The --safe-payloads option is now the default. Added
  --include-payloads for the special situations where payloads are
  needed. [Colin Rice]

o [NSE] Added new functionality and fixed some bugs in the brute library:
  - Added support for restricting the number of guesses performed by the
    brute library against users, to prevent account lockouts.
  - Added support to guess the username as password. The documentation
    previously suggested (wrongly) that this was the default behavior.
  - Added support to guess an empty string as password if not
    present in the dictionary. [Patrik]

o [NSE] Re-enabled support for guessing the username in addition to password
  that was incorrectly removed from the metasploit-xmlrpc-brute in previous
  commit. [Patrik]
  
o [NSE] Fixed bug that would prevent brute scripts from running if no service
  field was present in the port table. [Patrik]

o [NSE] Turned on promiscuous mode in targets-sniffer.nse so that it
  finds packets not only from or to the scanning host. [David]

o The Zenmap topology display feature is now disabled when there are
  more than 1,000 target hosts.  Those topology maps slow down the
  interface and are generally too crowded to be of much use.

o [NSE] Modified the http library to support servers that don't return valid
  chunked encoded data, such as the Citrix XML service. [Patrik]

o [NSE] Fixed a bug where the brute library would not abort even after all
  retries were exhausted [Patrik]

o Fixed a bug in the IPv6 OS probe called NI. The Node Information
  Query didn't include the target address as the payload, so at least
  OS X didn't respond. This differed from the probe sent by the
  ipv6fp.py program from which some of our fingerprints were derived.
  [David]

o [NSE] Fixed an error in the mssql library that was causing the
  broadcast-ms-sql-discover script to fail when trying to update port version
  information. [Patrik]

o [NSE] Added the missing broadcast category to the broadcast-listener script.
  [Jason DePriest]

o [NSE] Made changes to the categories of the following scripts (new
  categories shown) [Duarte Silva]:
  - http-userdir-enum.nse (auth,intrusive)
  - mysql-users.nse (auth,intrusive)
  - http-wordpress-enum.nse (auth,intrusive,vuln)
  - krb5-enum-users.nse (auth,intrusive)
  - snmp-win32-users.nse (default,auth,safe)
  - smtp-enum-users.nse (auth,external,intrusive)
  - ncp-enum-users.nse (auth,safe)
  - smb-enum-users.nse (auth,intrusive)

o Made nbase compile with the clang compiler that is a part of Xcode
  4.2. [Daniel J. Luke]

o [NSE] Fix a nil table index bug discovered in the mongodb
  library. [Thomas Buchanan]

o [NSE] Added XMPP support to ssl-cert.nse.

o [NSE] Made http-wordpress-enum.nse able to get names of users who
  have no posts. [Duarte Silva]

o Increased hop distance estimates from OS detection by one. The
  distance now counts the number of hops including the final one to
  the target, not just the number of intermediate nodes. The IPv6
  distance calculation already worked this way. [David]

And here is the download link again:

http://nmap.org/book/man-bugs.html

Enjoy the release, and don't forget to report any bugs found
(instructions: http://nmap.org/book/man-bugs.html).  My goal is to
make the next stable version this month.

Cheers,
Fyodor
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/