Re: [BUG] NSE/Nsock filehandle exhaustion

[Brandon Enright <bmenrigh () ucsd edu>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 30 Aug 2007 20:45:34 +0200
majek04 <majek04 () gmail com> wrote:

> On 8/28/07, Brandon Enright <bmenrigh () ucsd edu> wrote:
> > Maybe there needs to be some (tunable) cap on the number parallel NSE
> > scripts or number of sockets allowed open by NSE at a time.  I'm
> > thinking that nmap.new_socket() could block if the number of
> > connected/open sockets goes above some threshold.
> >
> > Brandon
>
> Hi!
>
> I implemented it exactly as you proposed. nsock_connect blocks when the
> number of opened connections exceeds some threshold.

Great, you got this patch out quick.

> For this moment this threshold is hardcoded  in
> #define NSOCK_DESCRIPTORS_THRESHOLD 768
> (btw. in future maybe it's good idea to decrease this value to 1 for
> easier nse debugging, you know, only one script running at a time)

Maybe the value should be set to what ever --max-parallelism is set to?
With Stoiko's patch, I can't get NSE to use more than about 50 Nsock
handles at once.  Obviously 768 is total overkill.  Rather than hardcode it
at some number, it would be nice to have it adjustable at runtime.

> My patch is rather hackish but it should work.

It is ;), it does :).


> (It can run two times lua_yield on lua thread, but it seems that it isn't
> bad, although it's not theoretically proper I think),

I haven't been able to get it to crash so with some tweaking and further
testing, should probably be included.

Your patch solves the Nsock handle problem in a totally different way than
Stoiko's patch but I don't think that means that one or the other
should be applied.  They BOTH should be applied.

With your patch, when the NSE loop starts, the NSOCK handles climb to 768
in a few seconds and stay pretty pegged there until the NSE scanning
finishes.  When this happens, the following lines are printed by the
thousands:

NSOCK: thread unqueued 0xa981f0
NSOCK: thread unqueued 0xa98ce0
NSOCK: thread unqueued 0xa995d0
NSOCK: thread unqueued 0xbb1130
NSOCK: thread unqueued 0xbb2030
<many, many more of these are printed>

Stoiko's patch takes care of these being printed unnecessarily by keeping
NSE below the NSOCK limit most of the time.

> Cheers!
>  Marek Majkowski
>
> (patch attached)


The one concern I have with this patch is what would happen if the limit
were set to 1 and then a single NSE script tried to make 2 sockets and
connect on both before doing anything else.  Am I right in assuming that
this script would deadlock because the script wouldn't be able to finish and
close the first without the second being made to allow it to continue?  Let
me know if this doesn't make sense or I'm just stupid.

Brandon


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFG1y6PqaGPzAsl94IRAm5CAJ9KJBOmOk6kmDperoaQWw1bwHffWwCeIZZF
UNUGWqTY3waPRJeKHzW6KiY=
=VqNN
-----END PGP SIGNATURE-----

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org