Re: Fathom 0.97 - Full Toolkit release, bug fixes, new features

[David Fifield <david () bamsoftware com>]

On Sun, Nov 07, 2010 at 05:31:51PM -0600, Tom Sellers wrote:
> Version 0.97 of the Fathom Toolkit has been released. The Fathom
> Toolkit is a suite of tools written with the goal of helping utilize
> Nmap to better understand your environment. The core tools are written
> in Ruby and leverage Kris Katterjohn's Nmap::Parser[1] Ruby library
> for searching and manipulating Nmap's XML output. Basic capabilities
> include querying large datasets for ports, services, OSes as well as
> providing metrics on the same.
>
>
> This version includes new scripts that round out the functionality of
> the suite. While previous releases primarily focused on querying
> existing XML formatted Nmap scan data, the new code handles the scan
> data's lifecycle including target management, recon, scanning,
> updating and purging.
>
> Release:      http://www.fadedcode.net/fathom/index.htm#Fathom0.97
> Changelog:    http://www.fadedcode.net/fathom/downloads.htm#Changelog
>
> New functionality:
>
>     * Addition of scan-full.sh, scan-recon.sh and scan-noping-full.sh
>       shell scripts. Each of these scripts performs a particular type
>       of single target scan. They can be executed manually for a
>       specific target, or called by the sweep scripts.

Fathom is a great example of how to run Nmap scans on a recurring basis.
I also appreciate the insight into how you do your scans.

This documentation is good (I was looking for it on the main page but
it's on the downloads page):

http://www.fadedcode.net/fathom/downloads.htm#BasicSetup

>       The scan output are files in *each* of Nmap's output formats
>       placed in the ./logs directory. There is ONE set of files PER
>       HOST. While this increases the number of files, it makes single
>       host updating and cleanup much simpler          
>
>     * Addition of sweep-full.sh and sweep-recon.sh shell scripts.
>       These scripts iterate over a list of hosts
>       (./lists/scanlist-random.txt) and execute either scan-full.sh or
>       scan-recon.sh. The scan output are files in *each* of Nmap's
>       output formats placed in the ./logs directory.
>
>     * Addition of update-data.sh shell script. This script rescans the
>       hosts in the ./logs directory with scan-full.sh starting with
>       the oldest first.
>
>     * Addition of fill-gaps.sh shell script. This script takes input
>       from ./lists/gaps.txt and scans the hosts with scan-recon.sh
>       ONLY if no files exist for the host in ./logs.
>
>     * Addition of util-genlist.sh shell script. This script takes a
>       list of target subnets from ./lists/subnets.txt and generates
>       two lists of targets: scanlist.txt and scanlist-random.txt.
>       scanlist-random.txt is the file that sweep-full.sh and
>       sweep-recon.sh use as their source of input.
>
>     * Addition of report.sh shell script. This script accepts an IP
>       address as input and simply echoes the contents of that IP's
>       .nmap file to the console if it exists. This simplifies quick
>       lookups of data for single hosts.

I like this report.sh option.

> Changes to prior functionality:
>
>     * fathom.rb - Added -m / --mac-address to search by MAC address or
>       MAC vendor string. This will use results from nbstat.nse if the
>       MAC data isn't present but nbstat data is. Thanks to Ron Bowes
>       (www.skullsecurity.org) for this idea.

This is a nice idea. This is another argument for better structured NSE
output. Scripts should be able to represent addresses and other data
without requiring special knowledge in tools like Fathom.

I was surprised at the results of this search:

$ ruby fathom.rb -m ab

192.168.0.190                                            00:16:CB:AE:D4:AC Apple Computer       2010/11/22 20:35:41

I don't see how "ab" matches anything there. Another example:

$ ruby fathom.rb -m ac

192.168.0.1                                              00:15:05:A2:C7:00 Actiontec Electronic 2010/11/22 20:50:24
192.168.0.190                                            00:16:CB:AE:D4:AC Apple Computer       2010/11/22 20:54:08

>     * util-cleanup.rb - Added IP address based selection of files to
>       move to the backup directory
>
>     * util-cleanup.rb - Added --purge command to delete backup
>       directory contents.
>
>     * Tabular (default) console output is much easier to read now.
>
>     * Fixed a issue in Fathom where --script-data was not searching
>       host script output.
>
>     * Misc fixes and enhancements can be found in the 0.97 changelog.
>
>
> All that being said, I have posted the information on Fathom on my site at
> http://www.fadedcode.net/fathom/
>
> For those of you that play around with or use Fathom I would greatly
> appreciate any and all feedback you feel like sending regardless of
> the topic (functionality, code quality, installation, site, etc).

I expected the Ruby and shell scripts to be executable. If that's
possible to do in a zip file it would be nice. The scripts even require
it:

# sh sweep-recon.sh
11/22/2010 08:49:49 PM Scanning 192.168.0.0
sweep-recon.sh: line 18: ./scan-recon.sh: Permission denied
11/22/2010 08:49:49 PM Scanning 192.168.0.1
sweep-recon.sh: line 18: ./scan-recon.sh: Permission denied
11/22/2010 08:49:49 PM Scanning 192.168.0.10
sweep-recon.sh: line 18: ./scan-recon.sh: Permission denied

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/