Re: [RFC] Some NSE optimizations

[David Fifield <david () bamsoftware com>]

On Tue, Jul 10, 2012 at 05:36:52PM -0500, Daniel Miller wrote:
> List,
>
> "Premature optimization is the root of all evil" I know, but I
> thought I'd throw NSE at a Lua profiler [1] and see if I could
> squeeze some more speed out of it. Granted also that our bottleneck
> is the network, but the changes I propose are fairly minor. Times
> below include profiling, which inflates them artificially, and I use
> a slow computer to start with. Patch attached.
>
> 1. nse_main.lua: tcopy(). This function gets called recursively on
> the host and port tables for each hostrule, portrule, and associated
> action functions. In my simple scan of one host, I saw 1402 calls
> with a total time of 1.9 seconds. I implemented it in C++ using
> lua_rawset, and saw a roughly 10x speedup (494 Lua function calls,
> the recursion is done in C++).
>
> 2. http.lua: skip_lws(). This function is called for each line in an
> HTTP header. In my default scan of one http service, it was called
> 712 times. I sped it up by using repeating pattern matching (* and
> +) instead of looping over repeated calls to string.match. This led
> to a 1.7x speedup, or .2 seconds on my 1-port scan.

Looks good.

> 3. http.lua: parse_header(). This function is called for each HTTP
> request (16 in my default scan), but took an astonishing 0.25
> seconds per call. It got a small speedup boost from skip_lws, which
> it calls repeatedly, but the primary speedup was using a single
> pattern match for non-whitespace instead of an explicit loop with 2
> string.match calls looking for whitespace on each character. This
> resulted in a 4.7x speedup, or about 3 seconds per http service.

> -    while pos <= #header and not string.match(header, "^\r?\n", pos) do
> -      s = pos
> -      while not string.match(header, "^[ \t]", pos) and
> -        not string.match(header, "^\r?\n", pos) do
> -        pos = pos + 1
> -      end
> -      words[#words + 1] = string.sub(header, s, pos - 1)
> -      pos = skip_lws(header, pos)
> +    while pos <= header_len and not string.match(header, "^\r?\n", pos) do
> +      s, e = string.find(header, "^[^%s]*", pos)
> +      words[#words + 1] = string.sub(header, s, e)
> +      pos = skip_lws(header, e+1)

I think it might be possible for this to enter an infinite loop,
depending on what %s matches. If there is a vertical tab, for example,
in the string, then the loop condition will always be true and the
string.find will always find a zero-length string. Since LWS can only be
space and horizontal tab, I think it suffices to be more narrow in the
pattern: "^[^ \t\r\n]*".

But I think this might still have problems with strings that contain a
\r not followed by \n. Would you test that and see that it works
correctly?

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/