Port memory bloat

[Pavel Kankovsky <kan () dcit cz>]

Nmap allocates one instance of Port per scanned port in hostgroup and this 
makes it devour enormous amounts of memory during a large-scale scan.

Let 1.2.3.0/24 be a network full of heavily firewalled hosts and let us 
consider the following command run on x86:

$ nmap -vv -sS -PN -p1- --min-hostgroup=256 1.2.3.1-254

(Lots of timing options are omitted for clarity. Large hostgroup size was 
intentional.)

When I ran it, it started at approximately 80 MB, grew to cca 180 MB during the 
course of the scanning and started bloating uncontrollably when Nmap started 
finishing hosts. It reached 1 GB and hit the vmsize limit I set to prevent 
damage to my computer when it had approximately 130 "hosts left".

When I restricted the scan to 10 hosts for the sake of the experiment:

$ nmap -vv -sS -PN -p1- --min-hostgroup=256 1.2.3.1-10

it grew to cca 20 MB during the scanning and bloated up to 80 MB
while the hosts were finished.

This makes approximately 7 MB per host or 100 bytes per scanned port and latter 
value matches the size of Port instance (on x86). Indeed, most of memory 
consumption growth occured when PortList::addPort() was called. Most of those 
100 bytes are wasted when the port is closed or filtered.

I split Port into two parts: Port with only the essential parts of the original 
Port and PortExt with the rest of fields, allocated on demand. Moreover I made 
minor changes to types and order of fields in Port and state_reason_t, and 
reduced the size of Port to 16 bytes (plus 4 bytes of malloc overhead).

These changes made the peak memory consumption of the aformentioned scans 
shrink to 500 MB (from 1+ GB) or 30 MB (from 80 MB) respectively.

See the attached patch against Nmap 5.00 but be warned that it has not been 
tested much.

PS: I am not subscribed to the list.
PS2: I sent the same message a week ago but it was probably lost.

--
Pavel Kankovsky

Attachment: nmap-5.00-bloat.patch
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org