Re: Follow up to NSE issues -- procedure to reproduce

[David Fifield <david () bamsoftware com>]

On Wed, Feb 25, 2009 at 09:12:10PM -0700, Patrick Donnelly wrote:
> On Wed, Feb 25, 2009 at 6:28 PM, Brandon Enright <bmenrigh () ucsd edu> wrote:
> > ==12614==
> > ==12614== Invalid read of size 8
> > ==12614==    at 0x58427C3: lua_pushboolean (in /usr/lib64/liblua.so.5.1.3)
> > ==12614==  Address 0xa5ab3c8 is 16 bytes inside a block of size 184 free'd
> > ==12614==    at 0x4C20A6A: free (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)
> > ==12614==    by 0x5850634: (within /usr/lib64/liblua.so.5.1.3)
> > ==12614==    by 0x584973B: (within /usr/lib64/liblua.so.5.1.3)
> > ==12614==    by 0x5847ECB: (within /usr/lib64/liblua.so.5.1.3)
> > ==12614==    by 0x58480F2: (within /usr/lib64/liblua.so.5.1.3)
> > ==12614==    by 0x58484C2: (within /usr/lib64/liblua.so.5.1.3)
> > ==12614==    by 0x5842CA7: lua_gc (in /usr/lib64/liblua.so.5.1.3)
>
> I'm fairly certain this is caused by nsock using the Lua thread after
> we have killed it (most likely due to timeout). Is there a way to stop
> nsock from doing work on behalf of the thread once we decide to
> destroy the thread?

I think you're quite right about the cause, and I have finally been able
to reproduce the problem. Brandon may also be right that there are other
causes besides timeouts. Besides the block of size 184 free'd there were
blocks of size 1,536.

The reproduction is tricky because of the timing involved. We must
synthesize this sequence of events:
  schedule NSE event → host timeout → garbage collection → NSE event callback
The event callback attempts to access memory freed by the garbage
collector.

The attached patch and script make the above sequence happen. The patch
forces a complete garbage collection every time a target times out and
makes the callback handler for stdnse.sleep access memory the way other
callbacks do. The script sleeps for 60 seconds if the target is
localhost, and 20 seconds otherwise. Run this command:

valgrind --suppressions=nmap_val.supp ./nmap --host-timeout 30s \
  -n -d --script=sleep localhost scanme.nmap.org

The port scan against localhost will be fast, and the scan against
scanme will be somewhat longer, using more of its alloted 30s which will
cause it to time out first during script scanning. Both scripts will
sleep, and scanme will time out before it wakes up and before localhost
times out. Then, before the localhost script times out too, the sleep
handler will be called and access invalid memory. The only purpose of
scanning localhost is to keep the script scan alive after scanme times
out.

==14793== Invalid read of size 4
==14793==    at 0x8125C47: lua_gettop (lapi.c:159)
==14793==    by 0x8110035: msevent_dispatch_and_delete (nsock_event.c:297)
==14793==    by 0x810E0D4: nsock_loop (nsock_core.c:913)
==14793==    by 0x80FD807: l_nsock_loop(int) (nse_nsock.cc:404)
==14793==    by 0x80F7872: process_mainloop(lua_State*) (nse_main.cc:465)
==14793==    by 0x80F8518: script_scan(std::vector<Target*, std::allocator<Target*> >&) (nse_main.cc:368)
==14793==    by 0x808D73E: nmap_main(int, char**) (nmap.cc:1822)
==14793==    by 0x8085441: main (main.cc:224)
==14793==  Address 0x4af1358 is 8 bytes inside a block of size 112 free'd
==14793==    at 0x4023BDA: free (vg_replace_malloc.c:323)
==14793==    by 0x8133127: l_alloc (lauxlib.c:631)
==14793==    by 0x812B4E2: luaM_realloc_ (lmem.c:79)
==14793==    by 0x812EE00: luaE_freethread (lstate.c:139)
==14793==    by 0x812AC7E: sweeplist (lgc.c:386)
==14793==    by 0x812AFB2: singlestep (lgc.c:583)
==14793==    by 0x812B396: luaC_fullgc (lgc.c:656)
==14793==    by 0x812668F: lua_gc (lapi.c:914)
==14793==    by 0x80F6A3A: process_finalize(lua_State*, unsigned int) (nse_main.cc:589)
==14793==    by 0x80F7B77: process_mainloop(lua_State*) (nse_main.cc:508)
==14793==    by 0x80F8518: script_scan(std::vector<Target*, std::allocator<Target*> >&) (nse_main.cc:368)
==14793==    by 0x808D73E: nmap_main(int, char**) (nmap.cc:1822)

So now what to do about it? We could store with each script thread a
list of pending Nsock event IDs, and cancel them all with
nsock_event_cancel whenever a process is finalized.

David Fifield

Attachment: nse-gc.diff
Description:

Attachment: sleep.nse
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org