printtableSize() causing huge spike in memory usage at end of hostgroup

[Brandon Enright <bmenrigh () ucsd edu>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Devs,

I've noticed for a while now that during the end of a hostgroup there
is a huge spike in Nmap's memory usage.  Generally this isn't an issue
but I have a box that is very pressed for memory and occasionally Nmap
runs it out right at the end of a huge scan.  Also, it seems that in
certain cases, NSE output can cause Nmap to waste a ton of memory right
and the end of the hostgroup.

In troubleshooting the bug Patrick pointed out with
printtableSize()/printableTable() the issue is obvious.

Nmap calculates the worst-case-scenario of memory usage.

That is, if your final output looks like this:

PORT     STATE SERVICE    REASON  VERSION
21/tcp   open  ftp        syn-ack WU-FTPD or MIT Kerberos ftpd 6.00LS
|_ banner: 220 www-xx.ucsd.edu FTP server (Version 6.00LS) ready.\x0D\x0A
22/tcp   open  ssh        syn-ack OpenSSH 4.5 (protocol 1.99)
|_ banner: SSH
|_ sshv1: Server supports SSHv1
|  ssh-hostkey: 1024 37:5e:70:5a:e5:9c:e7:ab:89:ab:61:94:d2:e6:85:f3 (RSA1)
|  35 12635613957272...sniped hundreds of bytes...123
|  1024 3e:a7:ef:cc:0c:ae:ca:59:71:fa:ca:9a:42:28:f3:74 (DSA)
...snip...

Where the SSH host key is hundreds of bytes long then Nmap will
calculate the memory needed for the output table to be the number of
rows in the table times the longest line (hundreds of bytes above).  It
doesn't matter that all the rest of the lines use just a handful of
bytes, the whole "rectangle" will be allocated.

Originally this wouldn't have been such a big problem because the
"PORT     STATE SERVICE    REASON  VERSION" table doesn't wast much
space because most lines are about the same (reasonable) length.

With NSE though, you could be penalized for having one extremely long
line printed.  I have a domain controller that really highlights the
problem.  If I use Ron's smb-enum-users.nse script on it, the script is
able to enumerate more than 100,000 accounts.  A few of those accounts
have really long account descriptions.  Before Nmap tries to print the
table the Nmap process uses about 4GB of ram.  As soon as the table
starts to be printed the memory usage balloons to more than 16GB of
ram.  This is because Nmap allocated enough memory for every line of
output to be extremely long.

At the cost of a little bit more CPU time, printableTable() can be
changed to dynamically realloc() memory as needed.  This would
eliminate the printtableSize() routine altogether.

I'm willing to submit a patch for this but I'd like to hear from
David/Fyodor/others before I do.

Brandon

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkm/EIEACgkQqaGPzAsl94LT4gCfRXqwqaDeEZZk3XaNkESqr66T
4TMAoJa5NaH/fNiWZz6147eGZy2P2x2/
=pKej
-----END PGP SIGNATURE-----

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org