Nmap Development mailing list archives
Re: [nmap-svn] r25684 - nmap-exp/calderon/nselib
From: Djalal Harouni <tixxdz () opendz org>
Date: Wed, 10 Aug 2011 14:17:27 +0100
On Tue, Aug 09, 2011 at 11:46:53PM -0400, Patrick Donnelly wrote:
Hi Paulino, On Tue, Aug 9, 2011 at 3:33 AM, <commit-mailer () insecure org> wrote:Author: paulino Date: Tue Aug 9 00:33:52 2011 New Revision: 25684 Log: improvements Modified: nmap-exp/calderon/nselib/httpspider.lua Modified: nmap-exp/calderon/nselib/httpspider.lua [...] +local function add_visited_uri(uri, page_obj) + VISITED_MUTEX "lock" + if nmap.registry[HTTPSPIDER_DATAKEY]["visited"][uri] == nil then + nmap.registry[HTTPSPIDER_DATAKEY]["visited"][uri] = page_obj end - VL_MUTEX "done" + VISITED_MUTEX "done" endThere's no need for this mutex here. Scripts (Lua threads -- coroutines) do not yield asynchronously; that is, they do not yield at arbitrary locations in the code. Mutexes in NSE are used to serialize access to some resource *when the script may yield between the lock and unlock*.
What Patrick tries to say here is that if your _coroutines_ access a share resource between some code that can yield then you need to protect this block of code with mutexes. In this function you should not use mutexes for one (or two reasons): 1) There are no calls to any function that can yield, if your code was: local function add_visited_uri(uri) VISITED_MUTEX "lock" if nmap.registry[...]["visited"][uri] == nil then local page_obj = socket:receive() nmap.registry[...]["visited"][uri] == page_obj end VISITED_MUTEX "done" end Then you can use mutexes, since scripts or coroutines will all yield at the socket:receive() call. 2) If there are calls to functions that can yield but you are sure that you are not processing some shared resources. In your example you are accessing the registry, however your code just fetches and stores page objects in the registry (no extra complex operations), so if you are absolutely sure that your workers will not pass the _same_ uri value to this _local_ add_visited_uri() function then you can avoid mutexes even if there are calls to functions that can yield. (this one is just to clarify :) )
Scripts only yield *explicitly* through calls to socket functions (e.g. socket:connect(...) or nmap.sleep(...)) or through the concurrency control mechanisms (mutexes and condition variables).
This is the list of functions that will make your code yield: * condvar("wait") * mutex("lock") -- will yield if the mutex is busy * stdnse.sleep() * socket:connect() - If too many threads have open sockets, we reach the max_parallelism 20 or Nmap's max_parallelism option. - If the connection succeed. * socket:reconnect_ssl() * socket:send() * socket:sendto() * socket:receive() * socket:receive_*() * pcap:pcap_receive() and coroutine.yield() Patrick should we add them to the scripting.xml doc ? -- tixxdz http://opendz.org _______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://seclists.org/nmap-dev/
Current thread:
- Re: [nmap-svn] r25684 - nmap-exp/calderon/nselib Patrick Donnelly (Aug 09)
- Re: [nmap-svn] r25684 - nmap-exp/calderon/nselib Djalal Harouni (Aug 10)