Nmap Development mailing list archives
hadoop and hbase information gathering
From: John Bond <john.r.bond () gmail com>
Date: Sat, 8 Oct 2011 15:48:14 +0200
I have written a couple of scripts to scrape various hadoop and hbase status pages. would be great to get feed back on wether this works with different version this is tested on hadoop 0.20. All scripts also implement newtargets where appropriate so you can run a command like the following to discover an entire hadoop cluster nmap --script hadoop-datanode-info,hbase-master-info,hadoop-jobtracker-info,hadoop-namenode-info,hadoop-tasktracker-info,hbase-region-info.nse --script-args hadoop-jobtracker-info.userinfo,newtargets -p 60010,50030,50070,50075,50060,60030 master-hadoop-server.example.com hadoop-jobtracker-info.nse information gathered: * State of the jobstracker * Date/Time the service was started * Hadoop Version * Hadoop Complie date * Jobtracker ID * Log Directory (reletive to the http://host:port/) * Associated Tasktrackers * Optionally also show the user activity history --- -- @usage -- nmap --script hadoop-jobtracker-info -p 50030 host -- -- @output -- 50030/tcp open hadoop-jobtracker -- | hadoop-jobtracker-info: -- | State: RUNNING -- | Started: Wed May 11 22:33:44 PDT 2011, bob -- | Version: 0.20.2 (f415ef415ef415ef415ef415ef415ef415ef415e) -- | Compiled: Wed May 11 22:33:44 PDT 2011 by bob from unknown -- | Identifier: 201111031342 -- | Log Files: logs/ -- | Tasktrackers: -- | tracker1.example.com:50060 -- | tracker2.example.com:50060 -- | Userhistory: -- | User: bob (Wed Sep 07 12:14:33 CEST 2011) -- |_ User: bob (Wed Sep 07 12:14:33 CEST 2011) -- --- hadoop-tasktracker-info.nse information gathered: * Hadoop Version * Hadoop Complie date * Log Directory (reletive to the http://host:port/) --- -- @usage -- nmap --script hadoop-tasktracker-info -p 50060 host -- -- @output -- PORT STATE SERVICE REASON -- 50060/tcp open hadoop-tasktracker syn-ack -- | hadoop-tasktracker-info: -- | Version: 0.20.1 (f415ef415ef415ef415ef415ef415ef415ef415e) -- | Compiled: Wed May 11 22:33:44 PDT 2011 by bob from unknown -- |_ Logs: /logs/ --- hadoop-namenode-info.nse information gathered: * Date/Time the service was started * Hadoop Version * Hadoop Complie date * Upgrades status * Filesystem Directory (reletive to the http://host:port/) * Log Directory (reletive to the http://host:port/) * Associated Datanodes --- -- @usage -- nmap --script hadoop-namenode-info -p 50070 host -- -- @output -- PORT STATE SERVICE REASON -- 50070/tcp open hadoop-namenode syn-ack -- | hadoop-namenode-info: -- | Started: Wed May 11 22:33:44 PDT 2011 -- | Version: 0.20.2-cdh3u1, f415ef415ef415ef415ef415ef415ef415ef415e -- | Compiled: Wed May 11 22:33:44 PDT 2011 by bob from unknown -- | Upgrades: There are no upgrades in progress. -- | Filesystem: /nn_browsedfscontent.jsp -- | Logs: /logs/ -- | Storage: -- | Total Used (DFS) Used (Non DFS) Remaining -- | 100 TB 85 TB 500 GB 14.5 TB -- | Datanodes (Live): -- | Datanode: datanode1.example.com:50075 -- | Datanode: datanode2.example.com:50075 hadoop-datanode-info.nse information gathered: * Log Directory (reletive to the http://host:port/) --- -- @usage -- nmap --script hadoop-datanode-info.nse -p 50075 host -- -- @output -- PORT STATE SERVICE REASON -- 50075/tcp open hadoop-datanode syn-ack -- | hadoop-datanode-info: -- |_ Logs: /logs/ hbase-master-info.nse information gathered: * Hbase Version * Hbase Complie date * Hbase Root Directory * Hadoop Version * Hadoop Complie date * Average Load * Zookeeper Quorum Server * Associated Region Servers --- -- @usage -- nmap --script hbase-master-info -p 60010 host -- -- @output -- | hbase-master-info: -- | Hbase Version: 0.90.1 -- | Hbase Compiled: Wed May 11 22:33:44 PDT 2011, bob -- | HBase Root Directory: hdfs://master.example.com:8020/hbase -- | Hadoop Version: 0.20 f415ef415ef415ef415ef415ef415ef415ef415e -- | Hadoop Compiled: Wed May 11 22:33:44 PDT 2011, bob -- | Average Load: 0.12 -- | Zookeeper Quorum: zookeeper.example.com:2181 -- | Region Servers: -- | region1.example.com:60030 -- |_ region2.example.com:60030 hbase-region-info.nse information gathered: * Hbase Version * Hbase Complie date * A bunch of metrics about the state of the region server * Zookeeper Quorum Server --- -- @usage -- nmap --script hbase-region-info -p 60030 host -- -- @output -- PORT STATE SERVICE REASON -- 60030/tcp open hbase-region syn-ack -- | hbase-region-info: -- | Hbase Version: 0.90.1 -- | Hbase Compiled: Wed May 11 22:33:44 PDT 2011, bob -- | Metrics requests=0, regions=0, stores=0, storefiles=0, storefileIndexSize=0, memstoreSize=0, -- | compactionQueueSize=0, flushQueueSize=0, usedHeap=0, maxHeap=0, blockCacheSize=0, -- | blockCacheFree=0, blockCacheCount=0, blockCacheHitCount=0, blockCacheMissCount=0, -- | blockCacheEvictedCount=0, blockCacheHitRatio=0, blockCacheHitCachingRatio=0 -- |_ Zookeeper Quorum: zookeeper.example.com:2181
Attachment:
hbase-region-info.nse
Description:
Attachment:
hbase-master-info.nse
Description:
Attachment:
hadoop-tasktracker-info.nse
Description:
Attachment:
hadoop-namenode-info.nse
Description:
Attachment:
hadoop-datanode-info.nse
Description:
Attachment:
hadoop-jobtracker-info.nse
Description:
_______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://seclists.org/nmap-dev/
Current thread:
- hadoop and hbase information gathering John Bond (Oct 08)
- RE:[NSE] hadoop and hbase information gathering John Bond (Oct 11)
- Re: hadoop and hbase information gathering David Fifield (Oct 12)
- Message not available
- Fwd: hadoop and hbase information gathering John Bond (Oct 12)
- Re: Fwd: hadoop and hbase information gathering David Fifield (Oct 13)
- Re: Fwd: hadoop and hbase information gathering Patrick Donnelly (Oct 13)
- Re: Fwd: hadoop and hbase information gathering Patrick Donnelly (Oct 13)
- Re: Fwd: hadoop and hbase information gathering John Bond (Oct 13)
- Re: Fwd: hadoop and hbase information gathering John Bond (Oct 30)
- Re: Fwd: hadoop and hbase information gathering David Fifield (Oct 31)
- Re: Fwd: hadoop and hbase information gathering David Fifield (Nov 08)
- Message not available