Nmap Development mailing list archives
luaffi for Nmap
From: Patrick Donnelly <batrick () batbytes com>
Date: Wed, 20 Jun 2012 00:37:47 -0400
In effort to clear up any confusion I may have caused in the IRC meeting [1], I want to lay out the situation more clearly. Especially, I want to make clear the Lua ecosystem: Lua is maintained by a committee of 3 people at PUC-Rio in Rio de Janeiro, Brazil [2]. It is a small development team that keeps development closed but source open. They do communicate quite frequently on the lua-l mailing list though. For the purposes of discussing different implementations of Lua, many use PUC-Lua to distinguish (as many use CPython for Python's main implementation). Still, PUC-Lua is the official Lua source and so plain "Lua" is more frequently used. I also want to make clear that Lua is "ANSI C" only with a very limited number of exceptions. These exceptions are only in libraries, not in the language core. The require function and dynamic shared library loading is one example of breaking ANSI C (in terms of using functions not offered by ANSI C). Lua's core design philosophy is to build the mechanisms, let the downstream providers do the rest. This is particularly true for libraries. LuaJIT [3] is a one-man project by the illustrious Mike Pall. LuaJIT is renowned for being the fastest (trace) JIT compiler for dynamic languages in existence. Besides being a great JIT compiler and interpreter (the interpreter alone is about 2-5 times faster than PUC-Lua), LuaJIT also offers a Foreign Function Interface (FFI) [4]. This was a solo effort Mike Pall took on that is unique for LuaJIT. The #1 goal of the project was to eliminate the overhead of Lua->C calls for bindings (e.g. NSE's nsock is one such binding). I emphasize there is no official ties between LuaJIT and PUC-Lua. In fact, LuaJIT has decided to forge it's own hybrid path with the release of Lua 5.2 by cherry picking features that Mike Pall decides are worth having while discarding the rest. This isn't necessarily a bad thing as Mike is 100% focused on Lua 5.1 support and speed. It does introduce fragmentation, however. LuaJIT's FFI library has been very well received by the Lua community. It has also been discussed widely in other forums. Because of its success, there have been efforts to create a fork [5, 6] to be used by PUC-Lua. [Keep in mind, as with all JIT compilers and third-party implementations of a language, not everyone is able to or willing to make the switch from the official language source. Nmap, in my opinion, has little to gain, currently, from switching to LuaJIT.] This library was, like most Lua bindings and LuaJIT, developed outside of the Lua team's influence (the 3 person committee). luaffi will probably never be incorporated into PUC-Lua. Still, we may be able to use luaffi in Nmap for our own purposes. I brought this topic up in IRC in order to expose another avenue to add support for ssh (via libssh). The three possible options initially discussed were (from IRC): 1) Integrate a 3rd party C library with acceptable license such as libssh2 2) Implement it in Lua ourselves or with Lua SSH library (which we haven't really been able to find, I don't think) 3) Do external call out to ssh binary, which avoids bloating Nmap with a library, but then it only works if they have ssh installed, and I guess on Windows we'd have to include an ssh.exe probably The first option is the writing of a C binding for NSE. We do this for nsock, pcrelib, and others. The second option is the usual approach (where feasible) of implementing the protocol in plain Lua. The third option is very unfavorable as it requires the presence of some executable (e.g. "ssh") and a common command line interface. Windows makes this option particularly difficult. I later brought up the fourth option which is luaffi. I believe it's a possible mechanism for binding libraries to Nmap using plain Lua code. It would definitely help with generating difficult scripts more quickly as the libraries for a particular protocol, e.g. ssh, can be quickly bound and used so the actual task (whatever that may be) can be achieved. [For example, the binding for the compression library zlib is quite small [7].] Best of all, there is no bloating of Nmap as these libraries can be conditionally bound at runtime. In fact, if the library is not available, a smart script will abort running at all without causing a fuss. Anyway, I hope I cleared up the confusion about this so people can make a more educated decision about what to do or not do with luaffi. There are still things to explore, such as stability of this luaffi fork. [1] http://seclists.org/nmap-dev/2012/q2/799 [2] http://www.lua.org/about.html [3] http://www.luajit.org/ [4] http://luajit.org/ext_ffi.html [5] http://lua-users.org/lists/lua-l/2011-07/msg00467.html [6] https://github.com/jmckaskill/luaffi [7] http://luajit.org/ext_ffi_tutorial.html#zlib -- - Patrick Donnelly _______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://seclists.org/nmap-dev/
Current thread:
- luaffi for Nmap Patrick Donnelly (Jun 19)
- Re: luaffi for Nmap David Fifield (Jun 22)