Re: [Zenmap-Patch] MemoryError

[Jay Bosamiya <jaybosamiya () gmail com>]

Hi All!

I decided to do some analysis on memory usage in Zenmap before and after
the patch.

The analysis is at [1] and the statistics are at [2].

I hope this analysis will not only help push this patch for commit but
also will help point any further work in the correct direction.

As far as I can see, the output that Nmap generates is repeated at
multiple places in Zenmap leading to unnecessarily high memory usage.
Getting rid of these repetitions would reduce the memory consumption.
Also, if we were to use a file based paging system, then this will no
longer be a problem.

Cheers,
Jay

Links:
[1]
https://github.com/jaybosamiya/Nmap-Tests/blob/master/Memory-Analysis/Analysis.md
[2]
https://github.com/jaybosamiya/Nmap-Tests/blob/master/Memory-Analysis/Statistics.md

On Saturday 07 June 2014 07:15 PM, Jay Bosamiya wrote:
> Hi All!
>
> We get many reports on the mailing list about Zenmap crashing with a
> MemoryError. Until we fix the underlying issue, we want to catch this
> bug and provide the user a more useful error message so that we get
> less of the out-of-memory "crash reports".
>
> If you want to see the bug for yourself:
>
>   * Obviously, if you have low RAM, then you just run Zenmap :P
>   * Running Zenmap on a system with lesser memory (or in a VM) is an
>     option; however this is cumbersome.
>   * Use scans that generate *HUGE* output. This usually takes too long
>     if you have enough RAM on your system.
>   * The best way I could see is to do the following:
>      1. Run |"softlimit -a 1000000000 python zenmap"| to limit the
>         memory that Zenmap can use (you may have to tweak the number
>         here; this value works well enough for me)
>      2. Use |"nmap 127.0.0.1/24 --packet-trace -p- -r|" as the Nmap
>         Command in the GUI. This generates extremely large output.
>      3. Run the scan and wait for a while
>      4. BOOM!
>   * Using "nmap 127.0.0.1/24 --packet-trace -p- -r" may be too slow to
>     actually wait for scan to complete, so you can use "nmap
>     --packet-trace -r --top-ports 1500 127.0.0.1/24" instead.
>
> Fix: The MemoryError is not localized to just the output window and
> multiple |try-except| wraps are needed. Each wrap caused one bug to
> disappear but allowed me to see the next.
> Finally, I was able to identify few key places where the try-except
> wraps are enough to catch the MemoryError. Attached is the patch for
> the same.
>
> Testing: I have tested each option multiple times and things seem to
> work as expected, with every MemoryError being caught. Some (very)
> rare crashes do occur, but that is because of the extremely
> restrictive environment that I am testing in, and they seem to
> disappear the moment I (even marginally) increase the memory I allow
> Zenmap to use.
>
> Comments are welcome, as always.
>
> Cheers,
> Jay

_______________________________________________
Sent through the dev mailing list
http://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/