Nmap Development mailing list archives
Re: Built-in authentication for http.lua
From: David Fifield <david () bamsoftware com>
Date: Thu, 12 Aug 2010 14:54:10 -0600
On Sat, Jul 31, 2010 at 10:42:14AM +0200, Patrik Karlsson wrote:
I implemented a quick brute script to test the new authentication feature and found two problems. One is related to caching and the other to calling the http lua with multiple worker threads. As my brute framework uses multiple worker threads to perform brute-forcing I noticed the following error popping up at times: ./nselib/http.lua:874: do not have a lock on this mutex stack traceback: [C]: in function 'mutex' ./nselib/http.lua:874: in function 'insert_cache' ./nselib/http.lua:1040: in function 'get' ./scripts/http-brute.nse:62: in function 'login' ./nselib/brute.lua:405: in function 'doAuthenticate' ./nselib/brute.lua:442: in function 'main' ./nse_main.lua:584: in function <./nse_main.lua:584> The error may be in how I'm calling the http lib from my code, I don't know. Anyway, when running with a single thread it works great. In order to test this change the line engine:setMaxThreads(1) in http-brute to 10.
I see this too. It seems to only happen at the end of the script. If I set unpwdb.timelimit=10s, it happens after 10 seconds, and with unpwdb.timelimit=20s, it happens after 20. I did some looking into this. The problem is that the http cache mutex is being freed twice. First by mutex_aux_done, which is defined in nse_nmaplib.cc and is "called when a thread ends but failed to unlock the mutex." It is then freed again in insert_cache by the http library. Patrick, do you have an idea why this is happening? I attached the patch that I used to diagnose it. The telltale parts are NSE: Finished http-brute against 192.168.0.1:80. mutex done aux_mutex_done 0x90c4990 mutex("done") mutex done insert_cache 0x90c3b20 mutex("done") NSE: Finished http-brute against 192.168.0.1:80. mutex done aux_mutex_done 0x90c25a8 mutex("done") mutex done insert_cache 0x90c4990 mutex("done") NSE: http-brute against 192.168.0.1:80 threw an error! ./nselib/http.lua:877: 0x90c4990 do not have a lock on this mutex To test this, it's reasonably easy to set up thttpd locally to require authentication, without installing anything. wget http://www.acme.com/software/thttpd/thttpd-2.25b.tar.gz tar xzvf thttpd-2.25b.tar.gz cd thttpd-2.25b perl -p -i -e 's/getline/_getline/g' extras/htpasswd.c ./configure && make ./extras/htpasswd -c .htpasswd test sudo ./thttpd -D Then you can reproduce this error by running ./nmap --datadir . localhost -p 80 -Pn -n --script http-brute -d --script-args http-brute.path=/,unpwdb.timelimit=5s
The cache problem occurs when a page is protected by authentication. As the cached result is not tied to the authenticated user any subsequent calls, regardless if they were authenticated or not will return the result. I don't know if this is a big problem for anything else, but when testing authentication it sure is, as all login attempts after the first successful will appear as correct. I worked around this in my script by issuing the no_cache option when calling the http.get method.
I hadn't thought of this. I had to make it so that 401 replies would not be cached, otherwise even the correct credentials would get the cached 401. I don't know what the right thing to do here is. Caching the authenticated response might be what someone wants. I think your idea of using no_cache is the best. David Fifield
Attachment:
mutex-done-debug.diff
Description:
_______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://seclists.org/nmap-dev/
Current thread:
- Built-in authentication for http.lua David Fifield (Jul 21)
- Re: Built-in authentication for http.lua David Fifield (Jul 25)
- Re: Built-in authentication for http.lua Patrik Karlsson (Jul 31)
- Re: Built-in authentication for http.lua David Fifield (Aug 12)
- Re: Built-in authentication for http.lua Patrick Donnelly (Aug 13)
- Re: Built-in authentication for http.lua Patrick Donnelly (Aug 13)
- Re: Built-in authentication for http.lua David Fifield (Sep 22)
- Re: Built-in authentication for http.lua Patrick Donnelly (Sep 22)
- Re: Built-in authentication for http.lua David Fifield (Sep 23)
- Re: Built-in authentication for http.lua Patrik Karlsson (Jul 31)
- Re: Built-in authentication for http.lua David Fifield (Jul 25)