Sergey. [Status report 12/17]

[Sergey Khegay <g.sergeykhegay () gmail com>]

[Report 12/17]
Hello Nmap,

Accomplishments:

- Properly configured configure.ac and Makefile.in, therefore
  configure, scripts to add libssh2 library to the build. As a
  dependency zlib was also added to the configuration.

  By default libssh2 and zlib are included in the build.

- Added and configured libssh2 and zlib projects to the nmap Visual
  Studio solution. Now we have baker's dozen projects in the latter.

  New projects added ned build targets that we probably do not need,
  but I am going to leave for now.

- For both changes above you can check my github repo, gsoc-ssh-autoconfig
  branch, or just use the following:

  git clone -b gsoc-ssh-autoconfig \
    https://github.com/sergeykhegay/nmap.git nmap-build-test

- As for the bug with libssh2 integration. As I said before the bug is
  a consequence of my attempt to resolve memory leak/consumption issue,
  when libssh2 sessions were not properly freed. Somehow Lua's garbage
  collector did not call session's gc function (actually called, but
  not for all sessions).

  So the bug now is a segmentation fault.
  On Linux it can happen during the ssh-brute scan.

  On Windows it happens during NSE script (ssh-brute) finalization
  stage (all accounts were tried and threads are finishing), no matter
  how many accounts I use.

- On Windows segmentation fault happens when some cclosure is freed:

  msvcr120d.dll!5b24d179()  Unknown
  [Frames below may be incorrect and/or missing, no symbols loaded for
msvcr120d.dll]
  [External Code]
  nmap.exe!l_alloc(void * ud, void * ptr, unsigned int osize, unsigned int
nsize) Line 1003 C
  nmap.exe!luaM_realloc_(lua_State * L, void * block, unsigned int osize,
unsigned int nsize) Line 86 C
> nmap.exe!freeLclosure(lua_State * L, LClosure * cl) Line 692  C
  nmap.exe!freeobj(lua_State * L, GCObject * o) Line 700  C
  nmap.exe!sweeplist(lua_State * L, GCObject * * p, unsigned int count)
Line 743  C
  nmap.exe!sweepstep(lua_State * L, global_State * g, int nextstate,
GCObject * * nextlist) Line 1030 C
  nmap.exe!singlestep(lua_State * L) Line 1068  C
  nmap.exe!luaC_runtilstate(lua_State * L, int statesmask) Line 1104  C
  nmap.exe!luaC_fullgc(lua_State * L, int isemergency) Line 1166  C
  nmap.exe!lua_gc(lua_State * L, int what, int data) Line 1055  C
  nmap.exe!luaB_collectgarbage(lua_State * L) Line 182  C
  nmap.exe!luaD_precall(lua_State * L, lua_TValue * func, int nresults)
Line 365  C
  nmap.exe!luaV_execute(lua_State * L) Line 1134  C
  nmap.exe!luaD_call(lua_State * L, lua_TValue * func, int nResults) Line
496 C
  nmap.exe!luaD_callnoyield(lua_State * L, lua_TValue * func, int nResults)
Line 506  C
  nmap.exe!lua_callk(lua_State * L, int nargs, int nresults, int ctx, int
(lua_State *, int, int) * k) Line 924 C
  nmap.exe!run_main(lua_State * L) Line 651 C++
  nmap.exe!luaD_precall(lua_State * L, lua_TValue * func, int nresults)
Line 365  C
  nmap.exe!luaD_call(lua_State * L, lua_TValue * func, int nResults) Line
495 C
  nmap.exe!luaD_callnoyield(lua_State * L, lua_TValue * func, int nResults)
Line 506  C
  nmap.exe!f_call(lua_State * L, void * ud) Line 942  C
  nmap.exe!luaD_rawrunprotected(lua_State * L, void (lua_State *, void *) *
f, void * ud) Line 144  C
  nmap.exe!luaD_pcall(lua_State * L, void (lua_State *, void *) * func,
void * u, int old_top, int ef) Line 727 C
  nmap.exe!lua_pcallk(lua_State * L, int nargs, int nresults, int errfunc,
int ctx, int (lua_State *, int, int) * k) Line 968 C
  nmap.exe!script_scan(std::vector<Target *,std::allocator<Target *> > &
targets, stype scantype) Line 809  C++
  nmap.exe!nmap_main(int argc, char * * argv) Line 2155 C++
  nmap.exe!main(int argc, char * * argv) Line 228 C++
  [External Code]


  Using the closure's address I traced it down to a closure with 6
  upvalues. I do not know how to find a function declaration using its
  memory address, but in the whole NSE engine's project there are only
  two closures that use 6 upvalues:


  /Users/sergey/Projects/github/nmap/nse_nsock.cc:
   1113    lua_setfield(L, -2, "__index");
   1114    for (i = top+1; i < top+1+6; i++) lua_pushvalue(L, i);
  >1115:   lua_pushcclosure(L, nsock_gc, 6);
   1116    lua_setfield(L, -2, "__gc");
   1117    lua_newtable(L);
   ....
   1122    lua_pushvalue(L, top+3); /* PCAP_SOCKET */
   1123    for (i = top+1; i < top+1+6; i++) lua_pushvalue(L, i);
  >1124:   lua_pushcclosure(L, pcap_gc, 6);
   1125    lua_setfield(L, top+3, "__gc");
   1126    lua_pop(L, 1); /* PCAP_SOCKET */


- I did not investigate the problem on Linux yet, because I couldn't
  compile Nmap with debug symbols. Probably because I'm working on
  MacOS. I get errors like:

  Undefined symbols for architecture x86_64:
  "_llvm_gcda_emit_arcs", referenced from:
      ___llvm_gcov_writeout in liblua.a(lauxlib.o)
      ___llvm_gcov_writeout in liblua.a(linit.o)
      ___llvm_gcov_writeout in liblua.a(lstate.o)
      ___llvm_gcov_writeout in liblua.a(lapi.o)
      ___llvm_gcov_writeout in liblua.a(lgc.o)
      ___llvm_gcov_writeout in liblua.a(ldo.o)
      ___llvm_gcov_writeout in liblua.a(lfunc.o)
      ...

  I will try to build on Ubuntu virtual machine and debug with gdb,
  which I will need to learn.


Goals:
o: Debug with gdb on Linux
o: Investigate why nse_libssh2's session was not freed properly in the
   initial implementation. I attempted this before, but debugging with
   several coroutines running is not an easy task.

P.S. Just noticed that I sent my previous report with wrong numbering,
so I resent it with proper attributes.

_______________________________________________
Sent through the dev mailing list
https://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/