Nmap Development mailing list archives
[PATCH] Comm is tokin' and passing to HTTP
From: Kris Katterjohn <katterjohn () gmail com>
Date: Mon, 08 Sep 2008 00:53:28 -0500
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hey everyone, I've added a few new options to the Comm NSElib which I liked while looking at the HTTP lib. These in-turn allow Comm to be used as a back-end for HTTP. Comm can now take four new options. The first two are simple: * conntimeout is for specifying a timeout only for the socket connect() * reqtimeout is for specifying a timeout only for the receive*()s and send()s The next two add the most new functionality: * buf is for specifying a delimiter for parsing the data into tokens with the lesser-used receive_buf() function. This is used at the same level as the "lines" and "bytes" options, but instead of limiting the amount of returned data, it returns a table of all of the tokenized data (element=token). * bufkeep is for specifying whether or not to keep the delimiter in the returned data. It defaults to false. This means Comm uses receive_buf() itself rather than the stdnse make_buffer() function which HTTP used. I've attached a patch which adds these options to Comm and uses it in HTTP. I've tested it against quite a few machines, with chunked encoding and not, and it all works fine for me. Please let me know what you think. Thanks, Kris Katterjohn -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQIVAwUBSMS91v9K37xXYl36AQLN1g/+PaVIwJCCERtihTL5/rCAxWa21I4XHEoZ x28Nt0QHC41j0nLUtxRCc7O0xwMhaUcdG5Fxfl6dip11lnruz38ZtSBUXwc16ptf v7dFlDdeJiNnVtfC6zaxsH0/dUyoo/78o4iGTl/gCIAQhvpNy9vwyQLGHEFmSRPz Xqa8Lyl+H9nWFeMpKhlY4knTpj49K6jvEbY1F1wxGG+gUY+aEgl1NaW8LOEompDL KGuIO6kfpG1Iu7+TswVYMy4KIVTF4DAaVE1MJ2g0VIVEOtAIN6GGJhfcI+H9eivA JQx1aTOyhOCE0/nH0vHDVD6YaZGHG3T0z9gombh57+rtkA4Uh5Eik4oYNXVcT7Dt uWDMRJ/KePPJ2FXaU4bKhOkD2TpKb6XOBSn6hb4OOmAfPHNtBgQDWZj9uCw6JoVI MUNFIMR8gbS5d6CW/LZDV+epner/Zb9pN7TEkTG+LyF2uuy7IS4ZBy1ELbJTBl7R sTTSCRulffbzYsUwiFTHny4W2d83KpNZXbB8O/HvboCnKiBrgFjbZEdqwpPxm9LI J5V8CXT2mrfHP19dqS9Rc4ky6m/3i1RlogNWjBLjU0IQOaAA2gzYj5e8WvvPLcqL attLgWScm9pbtIoe2RvmF/t0PvmDgg51TSvbjp/88y95jLJwAUzIqbN3/Kiy1xmY l0YPiLyWHLo= =EK8N -----END PGP SIGNATURE-----
Index: nselib/http.lua =================================================================== --- nselib/http.lua (revision 10021) +++ nselib/http.lua (working copy) @@ -14,7 +14,7 @@ module(... or "http",package.seeall) -require 'stdnse' +require 'comm' require 'url' -- @@ -125,34 +125,29 @@ end local result = {status=nil,header={},body=""} - local socket = nmap.new_socket() - local default_timeout = {} - if options.timeout then - socket:set_timeout( options.timeout ) - else - default_timeout = get_default_timeout( nmap.timing_level() ) - socket:set_timeout( default_timeout.connect ) - end - if not socket:connect( host, port, protocol ) then + local cto, rto + + -- Set connect and request timeouts + cto = options.timeout or get_default_timeout(nmap.timing_level()).connect + rto = options.timeout or get_default_timeout(nmap.timing_level()).request + + -- Comm options now + options = {proto=protocol, conntimeout=cto, reqtimeout=rto, buf="\r\n"} + + local status, response = comm.exchange(host, port, data, options) + + if not status then return result end - if not options.timeout then - socket:set_timeout( default_timeout.request ) - end - if not socket:send( data ) then - return result - end - local buffer = stdnse.make_buffer( socket, "\r\n" ) - local line, _ local header, body = {}, {} - -- header loop - while true do - line = buffer() - if (not line or line == "") then break end - table.insert(header,line) + -- Copy a separate header, removing it from response + while #response > 0 do + line = table.remove(response, 1) + if line == "" then break end + table.insert(header, line) end -- build nicer table for header @@ -189,30 +184,25 @@ -- if the server used chunked encoding we have to 'dechunk' the answer local counter, chunk_size counter = 0; chunk_size = 0 - while true do + while #response > 0 do if counter >= chunk_size then counter = 0 - chunk_size = tonumber( buffer(), 16 ) - if chunk_size == 0 or not chunk_size then break end + chunk_size = tonumber( table.remove(response, 1), 16 ) + if chunk_size == 0 or not chunk_size or #response == 0 then break end end - line = buffer() - if not line then break end + local line = table.remove(response, 1) counter = counter + #line + 2 table.insert(body,line) end else - while true do - line = buffer() - if not line then break end - table.insert(body,line) + for _, line in ipairs( response ) do + table.insert(body, line) end end - socket:close() result.body = table.concat( body, "\r\n" ) return result - end get_default_timeout = function( nmap_timing ) Index: nselib/comm.lua =================================================================== --- nselib/comm.lua (revision 10021) +++ nselib/comm.lua (working copy) @@ -5,12 +5,18 @@ -- via nmap.new_try().\n -- \n -- These functions can all be passed a table of options, but it's not --- required. The relevant indexes for this table are bytes, lines, proto --- and timeout. bytes is used to provide the minimum number of bytes required --- for a read. lines does the same, but for the minimum number of lines. --- proto is used to set the protocol to communicate with, defaulting to --- "tcp" if not provided. timeout is used to set the socket timeout (see --- the socket function set_timeout() for details). +-- required. The relevant indexes for this table are bytes, lines, buf, +-- bufkeep, proto, conntimeout, reqtimeout and timeout. bytes is used to +-- provide the minimum number of bytes required for a read. lines does the +-- same, but for the minimum number of lines. buf is used to set a delimiter +-- for which to separate the data into tokens with, and causes the functions +-- to return a table filled with these tokens as separate elements. bufkeep +-- is a boolean value specifying whether or not to keep the delimiter in the +-- returned buf data or not. proto is used to set the protocol to communicate +-- with, defaulting to "tcp" if not provided. timeout is used to set the socket +-- timeout (see the socket function set_timeout() for details). For specifying +-- only the connect() timeout, you can use conntimeout. For specifying just +-- the receive*() and send() timeouts, you can use reqtimeout. -- @author Kris Katterjohn 04/2008 module(... or "comm", package.seeall) @@ -33,18 +39,9 @@ -- they return is either the response from the host, or the error message -- from one of the previous calls (connect, send, receive*). -- --- These functions can be passed a table of options with the following keys: +-- If no "lines", "bytes" or "buf" option is specified, the functions attempt +-- to read as many bytes as possible. -- --- bytes: Specifies the minimum amount of bytes are to be read from the host --- lines: Specifies the minimum amount of lines are to be read from the host --- proto: Specifies the protocol to be used with the connect() call --- timeout: Sets the socket's timeout with nmap.set_timeout() --- --- If neither lines nor bytes are specified, the calls attempt to read as many --- bytes as possible. If only bytes is specified, then it only tries to read --- that many bytes. Likewise, it only lines if specified, then it only tries --- to read that many lines. If they're both specified, the lines value is used. --- ------ -- Makes sure that opts exists and the default proto is there @@ -74,21 +71,37 @@ local sock = nmap.new_socket() - if opts.timeout then - sock:set_timeout(opts.timeout) + if not opts.conntimeout then + opts.conntimeout = opts.timeout end + if opts.conntimeout then + sock:set_timeout(opts.conntimeout) + end + local status, err = sock:connect(target, port.number, opts.proto) if not status then return status, err end + if not opts.reqtimeout then + opts.reqtimeout = opts.timeout + end + + if opts.reqtimeout then + sock:set_timeout(opts.reqtimeout) + end + -- If nothing is given, specify bytes=1 so NSE reads everything - if not opts.lines and not opts.bytes then + if not opts.lines and not opts.bytes and not opts.buf then opts.bytes = 1 end + if opts.bufkeep == nil then + opts.bufkeep = false + end + return true, sock end @@ -98,6 +111,21 @@ if opts.lines then status, response = sock:receive_lines(opts.lines) return status, response + elseif opts.buf then + local token + + response = {} + + -- Start tokin' some tokens + while true do + status, token = sock:receive_buf(opts.buf, opts.bufkeep) + if not status then + break + end + table.insert(response, token) + end + + return (#response > 0), response end status, response = sock:receive_bytes(opts.bytes) Index: docs/scripting.xml =================================================================== --- docs/scripting.xml (revision 10021) +++ docs/scripting.xml (working copy) @@ -1650,13 +1650,22 @@ <para> These functions can all be passed a table of options, but it's not required. The relevant indexes for this table are <literal>bytes</literal>, <literal>lines</literal>, - <literal>proto</literal> and <literal>timeout</literal>. <literal>bytes</literal> - is used to provide the minimum number of bytes required for a read. <literal>lines</literal> - does the same, but for the minimum number of lines. If neither are provided, these - functions attempt to read as many bytes as are available. <literal>proto</literal> - is used to set the protocol to communicate with, defaulting to <literal>"tcp"</literal> if not provided. + <literal>buf</literal>, <literal>bufkeep</literal>, <literal>proto</literal>, + <literal>conntimeout</literal>, <literal>reqtimeout</literal> and <literal>timeout</literal>. + <literal>bytes</literal> is used to provide the minimum number of bytes required for a read. + <literal>lines</literal> does the same, but for the minimum number of lines. <literal>buf</literal> + is used to specify a delimiter with which to separate the data returned. Using + <literal>buf</literal> returns all of the data, but as a table with the delimited tokens as + elements. <literal>bufkeep</literal> is used to specify whether or not to keep the delimiter + in the table elements (it defaults to false). If none of <literal>lines</literal>, + <literal>bytes</literal> or <literal>buf</literal> options are provided, these functions + attempt to read as many bytes as are available. <literal>proto</literal> is used to set + the protocol to communicate with, defaulting to <literal>"tcp"</literal> if not provided. <literal>timeout</literal> is used to set the socket timeout (see the socket function - <literal>set_timeout()</literal> for details). + <literal>set_timeout()</literal> for details). For specifying only the <literal>connect()</literal> + timeout, you can use <literal>conntimeout</literal>. For specifying only the only the + <literal>receive*()</literal> and <literal>send()</literal> timeouts, you can use + <literal>reqtimeout</literal>. </para> <variablelist>
_______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://SecLists.Org
Current thread:
- [PATCH] Comm is tokin' and passing to HTTP Kris Katterjohn (Sep 07)
- Re: [PATCH] Comm is tokin' and passing to HTTP Sven Klemm (Sep 09)
- Re: [PATCH] Comm is tokin' and passing to HTTP Kris Katterjohn (Sep 09)
- Re: [PATCH] Comm is tokin' and passing to HTTP Sven Klemm (Sep 09)