Re: printtableSize() causing huge spike in memory usage at end of hostgroup

[David Fifield <david () bamsoftware com>]

On Tue, Mar 17, 2009 at 11:38:13PM +0000, Brandon Enright wrote:
> Attached is a patch to use memory a little bit more efficiently.  The
> buffer starts at 1k and any time it is too small it is doubled in
> size.  I did some testing that suggests for most scans 1k is probably
> enough.
>
> Right now:
> Standard Windows host + -sV: TABLE_MEM: size: 464, allocated: 961
> Standard Unix Server + -sV: TABLE_MEM: size: 667, allocated: 1925
> Windows Exchange + -sV: TABLE_MEM: size: 620, allocated: 1041
> Windows Exchange + -sV -sC: TABLE_MEM: size: 1050, allocated: 1910
> Extremely noisy host + -sV -sC: TABLE_MEM: size: 6372, allocated: 113435
> Extremely noisy host + -sV -sC -d3: TABLE_MEM: size: 3260062, allocated: 86514530
>
> With this patch:
> Standard Windows host + -sV: TABLE_MEM: size: 489, allocated: 1024
> Standard Unix Server + -sV: TABLE_MEM: size: 667, allocated: 1024
> Windows Exchange + -sV: TABLE_MEM: size: 620, allocated: 1024
> Windows Exchange + -sV -sC: TABLE_MEM: size: 1050, allocated: 2048
> Extremely noisy host + -sV -sC: TABLE_MEM: size: 4661, allocated: 8192
> Extremely noisy host + -sV -sC -d3: TABLE_MEM: size: 3260161, allocated: 4194304

What about a simple scan without -sV? 1024 seems a little high to me.
You can cause fewer reallocations to happen in some cases by doing the
reallocation like this:

  if (cell->strlength + p + 2 > tableoutsz) {
    tableoutsz = (cell->strlength + p + 2) * 2;
    tableout = (char *) safe_realloc(tableout, tableoutsz);
  }

How often is a table reused? Is it worth freeing the table's buffer
memory, or trimming it to size, do you think?

David Fifield

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org