Re: nmap 5 memory usage

[David Fifield <david () bamsoftware com>]

On Fri, Nov 20, 2009 at 05:29:26PM -0700, David Fifield wrote:
> On Sun, Nov 08, 2009 at 12:15:39PM +0200, Dimitrios Apostolou wrote:
> > Hello nmap hackers,
> >
> > I just run latest nmap on an old PC with 128MB RAM and a few other 
> > programs running. The system slowed down significantly and started  
> > swapping like crazy.
> >
> > I'm just reporting it here because it seems that system requirements have 
> > been significantly raised without anyone noticing (if this is intentional 
> > please just ignore me). By pmap'ing the process I got about 70MB of  
> > anonymous memory mappings. My command line was as simple as 'nmap -A  
> > $IPADDR'. What do you think?
>
> I tried this command with the Massif memory profiler. For me it grows to
> about 70 MB too. It turns out that almost all of this (over 80%) is from
> parsing the nmap-os-db file.
>
> http://www.bamsoftware.com/wiki/Nmap/Memory#a20091120
>
> The OS database has been growing, but the bigger cause is probably the
> increase in the size of each test value, which was increased from 128 to
> 256 in r11074 in November 2008. All test values are allocated the same
> amount of memory, which is very wasteful because most values a shorter
> than 10 bytes. This is another target of the memory use reduction
> campaign.

I've been working on making OS scan more efficient in terms of memory,
and I just merged my changes. There's a record of progress starting at

http://www.bamsoftware.com/wiki/Nmap/Memory#a20091120

I ran Massif before and after with the command "nmap -O", then added up
the parts that belong to OS scan. Before this merge the memory usage was
66,832,396 bytes, and after it was 2,815,061, a decrease of about 95%.
The biggest gain came from storing unique strings only once.

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/