Assertion error related to Lua garbage collection of nsock pcap objects

[Daniel Miller <bonsaiviking () gmail com>]

List,

Henri and I have been tracking down a bug that so far only I have been 
able to reproduce. The bug presents as an assertion error:

nmap: nsock_event.c:406: msevent_new: Assertion `msiod->state != 
NSIOD_STATE_DELETED' failed.

I have been able to narrow the circumstances down to a run with these 
arguments:

sudo ./nmap -d --datadir . --script sniffer-detect.nse -T5 
192.168.1.0/24 -sn -n

The issue is a call to nsock_pcap_read_packet (from nmap.pcap_receive() 
in NSE) on a socket/IOD that has been closed with nsi_delete. By setting 
a breakpoint on nsi_delete, I found it was being called from pcap_gc() 
in nse_nsock.cc. The nsock_iod object is kept as a weak value in the 
table KEY_PCAP, meaning that it can be garbage-collected when it is not 
reachable in any other way from Lua-land.

I think (wild speculation follows!) that the problem is that the 
nsock_iod objects are not reachable from Lua because they are part of a 
full userdata object, the nse_nsock_udata, and Lua can't look inside 
userdata objects (they are considered blobs of memory). Because they are 
not reachable from Lua except through the KEY_PCAP weak table, they are 
subject to garbage collection. If this is the case, I don't know why it 
hasn't affected more people.

If this is true, then one solution may be to store the entire 
nse_nsock_udata object as the value in KEY_PCAP, instead of just the 
nsock_iod. This would allow the garbage collector to see it from the Lua 
threads that have "socket" objects open. However, this would prevent 
nse_nsock_udata objects from sharing pcap IODs, which would remove a 
large efficiency and probably break things.

A different solution would be to make sure that the Lua garbage 
collector can reach the nsock_iod from the nse_nsock_udata. The garbage 
collector follows metatable entries, and userdata can have metatables, 
so setting an entry in the nse_nsock_udata's metatable would make this 
work. All nse_nsock_udata objects currently share a single metatable, so 
this would have to be done as a sub-table to the metatable, with keys 
that uniquely identify the nse_nsock_udata object. When 
socket:pcap_open() is called (l_pcap_open), a new entry in this table 
would be made to make the nsock_iod object reachable. The nsock_gc 
function handles finalization of collected nse_nsock_udata objects, and 
would be responsible for removing the object's entry (if any) in the 
shared metatable, making the nsock_iod object reachable by one less path.

This is all a bit beyond my knowledge, so I really need an expert eye 
(*cough*Patrick*cough*) to make sure my theories make sense before I 
start mucking with the code.

Dan
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/