Nmap Development mailing list archives
[NSE] Lua 5.2 Upgrade
From: Patrick Donnelly <batrick () batbytes com>
Date: Wed, 4 Apr 2012 01:43:38 -0400
Hello list, Over the past few months I've been working on a branch that upgrades NSE to use Lua 5.2. I'll start off by giving you the location of the branch so you can try it out: $ svn checkout --username guest --password "" https://svn.nmap.org/nmap-exp/patrick/nse-lua52 nse-lua52 $ cd nse-lua52 && ./configure --with-liblua=included && make $ ./nmap --datadir . --script safe localhost If you want to play with the Lua interpreter/compiler, you can simply compile (after making nmap) using: $ gcc -o liblua/lua liblua/lua.c liblua/liblua.a -lm -ldl $ gcc -o liblua/luac liblua/luac.c liblua/liblua.a -lm -ldl This upgrade brings numerous improvements. I've added a page to the wiki which lists the most relevant or important changes to script developers: https://secwiki.org/w/Nmap/Lua_5.2 I've included the bulk of that wiki page at the end of this email for archival purposes. Please try out the branch and look at the wiki page to get a feel for the changes. All feedback including "it works" is welcome! I would like feedback on socket:receive_buf in particular as the code is new but functionally the same. The nse_bitlib.cc code is deprecated for now (the 'bit' library). There may be some functional differences of significance between the old bit library and the new Lua 5.2 bit32 library. I feel this should be sorted out separate from this branch. == Wiki page as Plain Text == New features in Lua 5.2 of interest to script developers: o New hexadecimal escape sequence for strings. For example, instead of using decimal escape "\010" you can instead use "\x0a". o Lexical environments with _ENV [1]. Environments are now simply upvalues of a function and you may introduce a new environment into a scope by creating a new local named _ENV with a lifetime that expires at the end of the scope. For example: local print = print function foo () a = 5 do local _ENV = {} print(a) --> nil a = 6 print(a) --> 6 end print(a) --> 5 end You may also do things like: local print = print function foo (_ENV) print(a) end foo({a = 5}) --> 5 foo({a = 6}) --> 6 The old Lua 5.1 environments are completely gone. setfenv/getfenv are gone as a result. o Coroutines may yield across most C call boundaries. One of the simplest examples of such an error in 5.1: $ lua51 Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio
F = coroutine.wrap(function(...)return pcall(function(...) return coroutine.yield(...) end, ...) end)print(F("Hello"))
false attempt to yield across metamethod/C-call boundary This changes allows for coroutines in generic for loop iterators which yield outside of the for loop. For example: local function lines (socket) while true do coroutine.yield(socket:receive_buf("\r?\n", false)) end end for line in lines(socket) do --> do something end [The above example doesn't handle EOF or other errors.] o Lua 5.2 comes with a default bit library called "bit32". o Lua closures are now cached. The ramifications are best illustrated: $ lua51 Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio
for i = 1, 2 do print(function() end) end
function: 0x2104ad0 function: 0x2104fe0 $ lua52 Lua 5.2.0 Copyright (C) 1994-2011 Lua.org, PUC-Rio
for i = 1, 2 do print(function() end) end
function: 0x1d51070 function: 0x1d51070 Notice the closures are equivalent (and we could confirm equality using ==). Notice though: $ lua52 Lua 5.2.0 Copyright (C) 1994-2011 Lua.org, PUC-Rio
for i = 1, 2 do print(function() return i end) end
function: 0x796070 function: 0x796d30 This produces distinct closures because the upvalues are different. Namely, the i local is recreated on each iteration of the loop so each closure references a different i. It is an optimization to cache closures. This optimization lessens the performance impact of an anonymous function used in a loop. For example: -- Escape non-graphical characters function escape (str) return (str:gsub("[^%w%s%p]", function (a) return ("\\x%02x"):format(a:byte()) end)); end In Lua 5.1, if the above escape function is called in a loop, it would create a closure for the call to gsub each time. These become "garbage" because they are only used once. The way to correct this is to pull the function out into a local above the escape function: -- Lua 5.1 optimization for escape function do local function helper (a) return ("\\x%02x"):format(a:byte()) end function escape (str) return (str:gsub("[^%w%s%p]", helper)); end end In Lua 5.2, this is largely unnecessary as the closure is cached for future use. o Lua 5.2 now supports goto. Be sure the Raptor fences aren't out before use. ::forever:: goto forever The goto mechanism was added partially to solve the lack of a continue statement and nested break. for i = 1, 10 do if i % 2 == 0 then goto cont end ::cont:: end for i = 1, 10 do for j = 1, 10 do for k = 1, 10 do if i + j + k == 99 then goto done end end end end ::done:: o __pairs and __ipairs metamethods. You can now define how an object is iterated over using the pairs and ipairs functions. $ lua52 Lua 5.2.0 Copyright (C) 1994-2011 Lua.org, PUC-Rio
t = {1, 2, 3, a = 1} setmetatable(t, {__ipairs = pairs}) -- make ipairs work like pairs for i, v in ipairs(t) do print(i, v) end
1 1 2 2 3 3 a 1 [1] http://www.lua.org/manual/5.2/manual.html#2.2 -- - Patrick Donnelly _______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://seclists.org/nmap-dev/
Current thread:
- [NSE] Lua 5.2 Upgrade Patrick Donnelly (Apr 03)
- Re: [NSE] Lua 5.2 Upgrade Fyodor (Apr 09)
- Re: [NSE] Lua 5.2 Upgrade Matt Selsky (Apr 10)
- Re: [NSE] Lua 5.2 Upgrade Patrick Donnelly (Apr 10)
- Re: [NSE] Lua 5.2 Upgrade Fyodor (Apr 13)
- Re: [NSE] Lua 5.2 Upgrade David Fifield (May 02)
- Re: [NSE] Lua 5.2 Upgrade Patrick Donnelly (May 27)
- Re: [NSE] Lua 5.2 Upgrade Djalal Harouni (May 27)
- Re: [NSE] Lua 5.2 Upgrade Fyodor (Apr 09)