Nmap Development mailing list archives
[RFC] Some NSE optimizations
From: Daniel Miller <bonsaiviking () gmail com>
Date: Tue, 10 Jul 2012 17:36:52 -0500
List,"Premature optimization is the root of all evil" I know, but I thought I'd throw NSE at a Lua profiler [1] and see if I could squeeze some more speed out of it. Granted also that our bottleneck is the network, but the changes I propose are fairly minor. Times below include profiling, which inflates them artificially, and I use a slow computer to start with. Patch attached.
1. nse_main.lua: tcopy(). This function gets called recursively on the host and port tables for each hostrule, portrule, and associated action functions. In my simple scan of one host, I saw 1402 calls with a total time of 1.9 seconds. I implemented it in C++ using lua_rawset, and saw a roughly 10x speedup (494 Lua function calls, the recursion is done in C++).
2. http.lua: skip_lws(). This function is called for each line in an HTTP header. In my default scan of one http service, it was called 712 times. I sped it up by using repeating pattern matching (* and +) instead of looping over repeated calls to string.match. This led to a 1.7x speedup, or .2 seconds on my 1-port scan.
3. http.lua: parse_header(). This function is called for each HTTP request (16 in my default scan), but took an astonishing 0.25 seconds per call. It got a small speedup boost from skip_lws, which it calls repeatedly, but the primary speedup was using a single pattern match for non-whitespace instead of an explicit loop with 2 string.match calls looking for whitespace on each character. This resulted in a 4.7x speedup, or about 3 seconds per http service.
I'll be running a more comprehensive scan tonight to look for other candidates, but these were by far the standouts in terms of %time running (18%, 4.8%, and 37%, respectively). Please let me know if I introduced bugs!
Dan[1] I used LuaProfiler from http://luaprofiler.luaforge.net/, but had to modify it to work with Lua 5.2 (https://github.com/bonsaiviking/luaprofiler)
Attachment:
optimize1.patch
Description:
_______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://seclists.org/nmap-dev/
Current thread:
- [RFC] Some NSE optimizations Daniel Miller (Jul 10)
- Re: [RFC] Some NSE optimizations Daniel Miller (Jul 11)
- Re: [RFC] Some NSE optimizations Patrick Donnelly (Jul 11)
- Re: [RFC] Some NSE optimizations Daniel Miller (Jul 11)
- Re: [RFC] Some NSE optimizations David Fifield (Jul 16)