Re: Weird Crash - "WAITING_TO_RUNNING"

[David Fifield <david () bamsoftware com>]

On Wed, Nov 03, 2010 at 04:32:30PM -0600, Nathan wrote:
> On Wed, Nov 3, 2010 at 1:33 PM, Nathan <nathan.stocks () gmail com> wrote:
> > On Wed, Nov 3, 2010 at 11:29 AM, Patrick Donnelly <batrick () batbytes com> wrote:
> > > On Wed, Nov 3, 2010 at 1:07 PM, Nathan <nathan.stocks () gmail com> wrote:
> > > > "Is this on a 32-bit machine?" -- The hardware is 64-bit capable, but
> > > > the old version of linux on it is compiled in 32-bit, I think.  The OS
> > > > can address more than 4GB RAM, but the stuff running on it seems to
> > > > act very 32-bit.  Python's maxint is 32-bit, anyway:
> > > >
> > > > $ python
> > > > Python 2.5.2 (r252:60911, Oct 30 2008, 18:03:18)
> > > > [GCC 4.1.2 (Gentoo 4.1.2 p1.1)] on linux2
> > > > Type "help", "copyright", "credits" or "license" for more information.
> > > > > > > import sys
> > > > > > > sys.maxint
> > > > 2147483647
> > > >
> > > > $ uname -a
> > > > Linux scan5 2.6.23-gentoo-r9 #4 SMP Tue Oct 12 16:02:12 MDT 2010 i686
> > > > Intel(R) Xeon(R) CPU E5507 @ 2.27GHz GenuineIntel GNU/Linux
> > >
> > > Your processor (running as 32 bit) is using PAE (Physical Address
> > > Extension) to allow more than 4GB of RAM. Your process is still only
> > > allowed ~4GB of addressable RAM. This is why Nmap is crashing for you.
> >
> > Okay, I'll buy that that is the _symptom_ we're seeing.  I can't
> > imagine you're trying to imply that it's normal for an nmap scan of
> > one target to take that much RAM, though.  At 65,535 ports, 4GB gives
> > you 64KB PER PORT.  I could generate a small jpeg thumbnail of the
> > traffic sent to each port at that rate.
> >
> > The question is, WHAT is using up that RAM, and how do we stop it?   I
> > could run multiple instances of this same scan on this same box with
> > nmap 4.68 (that I just upgraded from) against this target without any
> > problems at all.
> >
> > Did anyone look at the stack trace???  It seriously can't be normal to
> > run "skypev2-version" and "jdwp-version" tens of thousands of times
> > until you run out of RAM.  Infinite loop, anyone?  Or, perhaps this is
> > caused by nmap erroneously thinking every single port is open (which
> > is one of the things I hoped upgrading would fix), where the
> > underlying assumption is that very few ports will be open so we can
> > use a ton of memory to investigate each open port?
>
> Found a workaround:  add "--scan-delay 1ms".  It finishes a lot
> quicker too.  It seems much more accurate too.  At least it has a lot
> less than 65,535 ports listed as open like nmap 4.68 thought.
>
> So now my question is: According to the nmap man page, nmap will "try"
> to dynamically determine an appropriate delay time, but it's unclear
> as to whether specifying this option overrides that dynamic
> determination completely or if it just sets a starting point that will
> still be dynamically adjusted if it thinks is necessary.  Does anyone
> know which behavior I just described matches reality?

It is a hard limit that doesn't change throughout the scan.

The memory exhaustion you've found is clearly a bug in Nmap. If no one
has responded constructively it's because we don't know what might be
the cause yet.

Your discovery of --scan-delay is a good clue. Also the fact that ports
are being erroneously marked open. I can't think of a reason why
scanning faster would cause ports to be seen as open; Nmap never marks a
port open unless it gets some kind of response, and rate-limited replies
would show up as filtered instead.

Try doing your scan with fewer ports at full speed and see if they are
wrongly marked open. Like this:

/usr/bin/sudo /usr/bin/nmap -sS -sV -T4 --top-ports 100 74.62.92.70 -P0 -v

You should not get more open ports than you got with --scan-delay. If
you get more open ports, there's something weird going on that we have
to figure out. Run with the -v option and you'll be notified of open
ports in real time; that way you'll be able to see if you're getting a
flood of them.

For what it's worth, I just tried the scan and I am getting a flood of
them:

Discovered open port 30749/tcp on 74.62.92.70
Discovered open port 48748/tcp on 74.62.92.70
Discovered open port 8235/tcp on 74.62.92.70
Discovered open port 46066/tcp on 74.62.92.70
Discovered open port 16372/tcp on 74.62.92.70
Discovered open port 24221/tcp on 74.62.92.70
Discovered open port 20118/tcp on 74.62.92.70
Discovered open port 21920/tcp on 74.62.92.70
Discovered open port 47846/tcp on 74.62.92.70
Discovered open port 34008/tcp on 74.62.92.70
Discovered open port 59467/tcp on 74.62.92.70
Discovered open port 25711/tcp on 74.62.92.70
Discovered open port 47129/tcp on 74.62.92.70
Discovered open port 48583/tcp on 74.62.92.70
Discovered open port 65477/tcp on 74.62.92.70
Discovered open port 40840/tcp on 74.62.92.70
Discovered open port 42922/tcp on 74.62.92.70
Discovered open port 31149/tcp on 74.62.92.70
Discovered open port 37470/tcp on 74.62.92.70
Discovered open port 55904/tcp on 74.62.92.70
Discovered open port 55122/tcp on 74.62.92.70
Discovered open port 6422/tcp on 74.62.92.70
Discovered open port 987/tcp on 74.62.92.70

This would definitely be the cause of the memory consumption, if the
scripting engine is starting instances for all of these ports.

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/