[NSE] Optimize stdnse.format_output()

[Djalal Harouni <tixxdz () opendz org>]

I've attached a patch that will make the format_output_sub() function
used by the stdnse.format_output() to store results in a table and to
call table.concat() when returning instead of doing classic
concatenations. This will prevent stressing the Lua garbage-collector,
and it will speed up the process.

The table.concat() will first create a stack of strings, then it will
concat them using the Tower of Hanoi [1].


This is the output (end of the output) of a simple post-processing test
script (after patch applied):
|   Host:  (10.0.2.22)
|   Vuln X 341
|   Vulnerability Reported by: vulns-test-1
|     State: VULNERABLE
|     IDs:  NMAP:341
|     Description:
|       Generated vulnerability
|     References:
|       http://scanme.org/id?341
|   
|_  printed:  64320 entry
No data files read.
Nmap done: 64 IP addresses (64 hosts up) scanned in 279.33 seconds

During the scan some test scripts have added multiple vulnerabilities
entries, one of them was adding 1000 vulnerability entry for each host.

Without the patch if we specify '/28' mask then the current
implementation can eat all the memory when reporting all the
post-processing results (I've tested and confirmed this in a guest running
under Qemu). So I would like to commit the patch since it can have a huge
performance boost.


Just to note: I'm trying to optimize the vulnerability library, so sorry
for the delay, and I'll post more info about it later.

Thanks

[1] http://www.lua.org/pil/11.6.html

-- 
tixxdz
http://opendz.org

Attachment: stdnse.format_output_concat.diff
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/