Re: [NSE] Child Coroutine Patch with Explanation

[David Fifield <david () bamsoftware com>]

On Thu, Jun 04, 2009 at 05:46:53PM -0600, Patrick Donnelly wrote:
> When we run the base thread, remember that it may create its own
> coroutines that it may then run. If this coroutine were to make an
> nsock function call or a blocking call on a mutex, the yield,
> initiated by NSE, would incorrectly go to the base coroutine and not
> NSE itself. Observe here (nse_main.lua):

I am sure I am not the only one who has had trouble understanding this
patch and the motivation for it. I think that the patch should be
applied. I'm going to restate the issue in my own words to be sure I
understand.

The issue is related to the way NSE does parallelism. Really, the
parallelism is an illusion--only one script runs at a time, and it runs
until it decides to give up control. (Internally, each script is a Lua
coroutine, and it runs until it yields.) How do several scripts run at
once, given that a script never explicitly yields? The answer is that
each network function call, like connect, implicitly yields control back
to NSE. Other scripts may run while the connect is in progress; when it
finishes; NSE restores the yielded script by resuming its coroutine. For
the script it is transparent. The call to connect works just like a
normal function call.

The thing is that may create its own coroutines, and these coroutines
may call a network function such as connect. It hasn't been a problem
because none of our script to this. But when it happends, the network
call yields not back to NSE, but to the coroutine that created it--the
script itself.

What was supposed to look like a simple function call has now bizarrely
turned into an instant return to the main script coroutine. The connect
hasn't finished, but the script carries on anyway. NSE is still going to
initiate a connection on behalf of the coroutine. When the connection is
established, it will look for the correct script coroutine to resume,
but because it wasn't a script coroutine that asked for the connection,
NSE will throw away the result in confusion.

Patrick's patch restores the illusion of a function call for network
operations within a script coroutine. It makes it so that when a
coroutine yields, it and all its parents, all the way up to the script
level, yield also. When the network operation is finished, all those
coroutine are restored to their previous state.

It's a subtle problem, but still a bug. I think that there would have
been something like this patch in an earlier version of NSE if the
problem had been thought of then. The patch is short, but I want to read
it again so I understand how it works.

David Fifield

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org