RE:[NSE] hadoop and hbase information gathering

[John Bond <john.r.bond () gmail com>]

Have also added a script for the secondary namenode.  All these
scripts are also now in git
https://github.com/b4ldr/nse-scripts


information gathered:
 * Date/Time the service was started
 * Hadoop Version
 * Hadoop Complie date
 * hostname/ipaddress:port of the master namenode server
 * Last time a checkpoint was taken
 * How often checkpoints are taken (in seconds)
 * File size of current checkpoint

---
-- @usage
-- nmap --script  hadoop-secondary-namenode-info -p 50090 host
--
-- @output
-- PORT      STATE  SERVICE REASON
-- 50090/tcp open   unknown syn-ack
-- | hadoop-secondary-namenode-info:
-- |   Start: Wed May 11 22:33:44 PDT 2011
-- |   Version: 0.20.2, f415ef415ef415ef415ef415ef415ef415ef415e
-- |   Compiled: Wed May 11 22:33:44 PDT 2011 by bob from unknown
-- |   namenode: namenode1.example.com/193.0.19.125:8020
-- |   Last Checkpoint: Wed May 11 22:33:44 PDT 2011
-- |   Checkpoint Period: 3600 seconds
-- |_  Checkpoint Size: 12345678 MB
--
---------- Forwarded message ----------
From: John Bond <john.r.bond () gmail com>
Date: 8 October 2011 15:48
Subject: hadoop and hbase information gathering
To: nmap-dev <nmap-dev () insecure org>


I have written a couple of scripts to scrape various hadoop and hbase
status pages.  would be great to get feed back on wether this works
with different version this is tested on hadoop 0.20.  All scripts
also implement newtargets where appropriate so you can run a command
like the following to discover an entire hadoop cluster

nmap  --script 
hadoop-datanode-info,hbase-master-info,hadoop-jobtracker-info,hadoop-namenode-info,hadoop-tasktracker-info,hbase-region-info.nse
--script-args hadoop-jobtracker-info.userinfo,newtargets -p
60010,50030,50070,50075,50060,60030 master-hadoop-server.example.com



hadoop-jobtracker-info.nse

information gathered:
 * State of the jobstracker
 * Date/Time the service was started
 * Hadoop Version
 * Hadoop Complie date
 * Jobtracker ID
 * Log Directory (reletive to the http://host:port/)
 * Associated Tasktrackers
 * Optionally also show the user activity history

---
-- @usage
-- nmap --script hadoop-jobtracker-info -p 50030 host
--
-- @output
-- 50030/tcp open  hadoop-jobtracker
-- | hadoop-jobtracker-info:
-- |   State: RUNNING
-- |   Started: Wed May 11 22:33:44 PDT 2011, bob
-- |   Version: 0.20.2 (f415ef415ef415ef415ef415ef415ef415ef415e)
-- |   Compiled: Wed May 11 22:33:44 PDT 2011 by bob from unknown
-- |   Identifier: 201111031342
-- |   Log Files: logs/
-- |   Tasktrackers:
-- |     tracker1.example.com:50060
-- |    tracker2.example.com:50060
-- |   Userhistory:
-- |     User: bob (Wed Sep 07 12:14:33 CEST 2011)
-- |_    User: bob (Wed Sep 07 12:14:33 CEST 2011)
-- ---

hadoop-tasktracker-info.nse

information gathered:
 * Hadoop Version
 * Hadoop Complie date
 * Log Directory (reletive to the http://host:port/)

---
-- @usage
-- nmap --script hadoop-tasktracker-info -p 50060 host
--
-- @output
-- PORT      STATE SERVICE            REASON
-- 50060/tcp open  hadoop-tasktracker syn-ack
-- | hadoop-tasktracker-info:
-- |   Version: 0.20.1 (f415ef415ef415ef415ef415ef415ef415ef415e)
-- |   Compiled: Wed May 11 22:33:44 PDT 2011 by bob from unknown
-- |_  Logs: /logs/
---

hadoop-namenode-info.nse
information gathered:
 * Date/Time the service was started
 * Hadoop Version
 * Hadoop Complie date
 * Upgrades status
 * Filesystem Directory (reletive to the http://host:port/)
 * Log Directory (reletive to the http://host:port/)
 * Associated Datanodes

---
-- @usage
-- nmap --script hadoop-namenode-info -p 50070 host
--
-- @output
-- PORT      STATE SERVICE         REASON
-- 50070/tcp open  hadoop-namenode syn-ack
-- | hadoop-namenode-info:
-- |   Started:  Wed May 11 22:33:44 PDT 2011
-- |   Version:  0.20.2-cdh3u1, f415ef415ef415ef415ef415ef415ef415ef415e
-- |   Compiled:  Wed May 11 22:33:44 PDT 2011 by bob from unknown
-- |   Upgrades:  There are no upgrades in progress.
-- |   Filesystem: /nn_browsedfscontent.jsp
-- |   Logs: /logs/
-- |   Storage:
-- |   Total       Used (DFS)      Used (Non DFS)  Remaining
-- |   100 TB      85 TB           500 GB          14.5 TB
-- |   Datanodes (Live):
-- |     Datanode: datanode1.example.com:50075
-- |     Datanode: datanode2.example.com:50075

hadoop-datanode-info.nse

information gathered:
 * Log Directory (reletive to the http://host:port/)

---
-- @usage
-- nmap --script hadoop-datanode-info.nse -p 50075 host
--
-- @output
-- PORT      STATE SERVICE         REASON
-- 50075/tcp open  hadoop-datanode syn-ack
-- | hadoop-datanode-info:
-- |_  Logs: /logs/


hbase-master-info.nse

information gathered:
 * Hbase Version
 * Hbase Complie date
 * Hbase Root Directory
 * Hadoop Version
 * Hadoop Complie date
 * Average Load
 * Zookeeper Quorum Server
 * Associated Region Servers

---
-- @usage
-- nmap --script hbase-master-info -p 60010 host
--
-- @output
-- | hbase-master-info:
-- |   Hbase Version: 0.90.1
-- |   Hbase Compiled: Wed May 11 22:33:44 PDT 2011, bob
-- |   HBase Root Directory: hdfs://master.example.com:8020/hbase
-- |   Hadoop Version: 0.20  f415ef415ef415ef415ef415ef415ef415ef415e
-- |   Hadoop Compiled: Wed May 11 22:33:44 PDT 2011, bob
-- |   Average Load: 0.12
-- |   Zookeeper Quorum: zookeeper.example.com:2181
-- |   Region Servers:
-- |     region1.example.com:60030
-- |_    region2.example.com:60030

hbase-region-info.nse

information gathered:
 * Hbase Version
 * Hbase Complie date
 * A bunch of metrics about the state of the region server
 * Zookeeper Quorum Server

---
-- @usage
-- nmap --script hbase-region-info -p 60030 host
--
-- @output
-- PORT      STATE SERVICE      REASON
-- 60030/tcp open  hbase-region syn-ack
-- | hbase-region-info:
-- |   Hbase Version: 0.90.1
-- |   Hbase Compiled: Wed May 11 22:33:44 PDT 2011, bob
-- |   Metrics requests=0, regions=0, stores=0, storefiles=0,
storefileIndexSize=0, memstoreSize=0,
-- |   compactionQueueSize=0, flushQueueSize=0, usedHeap=0, maxHeap=0,
blockCacheSize=0,
-- |   blockCacheFree=0, blockCacheCount=0, blockCacheHitCount=0,
blockCacheMissCount=0,
-- |   blockCacheEvictedCount=0, blockCacheHitRatio=0,
blockCacheHitCachingRatio=0
-- |_  Zookeeper Quorum: zookeeper.example.com:2181

Attachment: hbase-region-info.nse
Description:

Attachment: hbase-master-info.nse
Description:

Attachment: hadoop-tasktracker-info.nse
Description:

Attachment: hadoop-namenode-info.nse
Description:

Attachment: hadoop-datanode-info.nse
Description:

Attachment: hadoop-jobtracker-info.nse
Description:

Attachment: hadoop-secondary-namenode-info.nse
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/