New RPC Grinding implementation

[Hani Benhabiles <kroosec () gmail com>]

Hi list,

TL;DR: RPC Grinding NSE based implementation with improved performances.
14 years old code removed in the process.

When a RPC service is detected, Nmap's RPC grinder is used to 
fingerprint it.
This works by sending RPC NULL commands with varying program numbers 
from the
nmap-rpc list that comes with Nmap in order to determine the actual service
program and version.

Until last year, the scan was activated using the -sR flag, but is now 
combined
with -sV. For more information about RPC grinding, see [1].

Me and Henri have been working lately on replacing the current RPC grinding
implementation with a new NSE based one. The work aims to modernize the
scanning process, write easily maintainable code and benefit from some
performance improvements. You could find the new rpc-grind.nse script 
[2] and
the related changes in the rpc-grind branch [3].  Notice that in order 
to run the
script, the latest version of rpc.lua from nmap trunk is needed, which 
allows to
send commands with program numbers / versions that are not supported in the
library.

The current RPC grinding implementation is the last scan type that still 
uses the
(very) old pos_scan scan engine, so this is an extra incentive for us to 
kill some
code from 1998. :)

So far, we have done many tests that showed positive results.
We would like to see more tests on two aspects of the new implementation:
Accuracy and performance.

Testing for accuracy:
For this, we have rpc-grind-accuracy.nse [4] which is a small variation of
rpc-grind.nse. Instead of setting the port version values, it outputs it
alongside the results from the actual rpc grind (given that it was run 
with -sV.)

| rpc-grind-accuracy:
|                   NSE             OLD
| Name        rpcbind        rpcbind
| Number    100000        100000
| High ver    2                  2
|_Low ver    2                  2

If the results from the script and the current implementation didn't match,
it adds "WARNING: RPC Grinding results not matching." to the output.
This would allow us to grep through results from large network scans and
detect any problems, easily.

Testing for performance:
In essence, we would like to gather more info on scans in different 
scenarios:
directly connected hosts, internet scans, N machines with 1 rpc service 
each, 1
machine with N rpc services etc,. So far, we have had some very positive
results, as could be seen in the attached graphs. The attached 
reproducer script
makes it easier to automate simulating N services running on a machine.
You may need to modify values such as MAX_SERVICES, OUTFILE, ./nmap, 
./ncat/ncat, 111 and 8000
in the script depending on your environment and testing scenario.

Even if you don't happen to have a large variety RPC services laying out 
there,
some interesting tests could be done by changing the position of the 
program/number
pair in the nmap-rpc file and using --datadir to specify the modified 
nmap-rpc.
This would help testing scenarios with varying number of requests needed 
to detect
the according program number.

As for code metrics: here are some collected numbers on added and 
removed lines, compared
to the latest trunk revision.

global_structures.h: +1 -1
Makefile: +1 -1
nmap.cc: -9
nmap_rpc.cc: -691 (Removed completely)
nmap_rpc.h: -182 (Removed completely)
nse_nmaplib.cc: -14
output.cc: +3 -47
portlist.cc: +1 -59
portlist.h: +4 -21
scan_engine.h: +2 -6
scan_engine.cc:  +5 -340
docs/scripting.xml: -20
scripts/rpc-grind.nse: +261
Total:
+278 -1391

Feedback and tests are very welcome. Please, let us know if you come 
across any issues.

[1] http://nmap.org/book/vscan-post-processors.html#version-detection-rpc
[2] https://svn.nmap.org/nmap-exp/kroosec/rpc-grind/scripts/rpc-grind.nse
[3] https://svn.nmap.org/nmap-exp/kroosec/rpc-grind/
[4] 
https://svn.nmap.org/nmap-exp/kroosec/rpc-grind/scripts/rpc-grind-accuracy.nse

Cheers,
Hani.

--
Hani Benhabiles

Twitter: https://twitter.com/#!/kroosec
Blog: http://kroosec.blogspot.com

Attachment: measure_rpcg.sh
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/