Nmap Development mailing list archives
Nsock read buffer
From: David Fifield <david () bamsoftware com>
Date: Mon, 15 Nov 2010 22:23:15 -0800
On Wed, Sep 15, 2010 at 10:28:46PM -0500, Kris Katterjohn wrote:
On 09/15/2010 10:00 PM, David Fifield wrote:I would like to eventually remove receive_buf completely. I believe that such a function should be built into Nsock itself. There should be a buffer associated with each iod so that you can ask for a certain number of bytes (or lines, etc.) and get exactly that many, no more, with those left over remaining in the buffer. The ssh pattern would be packet_size = socket:receive_exactly(4) -- Unpack size. packet = socket:receive_exactly(packet_size) It would be an error if either call failed; they would return an error if they can't supply enough bytes. There would also be a call to return everything available on the socket, clearing the buffer. There is such a socket buffer (not using Nsock) in ncat/http.c. The main implementation difficulty that I have considered is that you can't just use the select function to detect when an iod has input pending, you also have to check if there is anything in the buffer. We would also need new name for the new "exact" functions. There are a lot of script that use receive_bytes(1) that expect to get as many bytes as are available.This involves changing the behavior of existing functions but I think is a better solution (or at least the beginning of one): receive(): Give me whatever is available receive_bytes(n): Give me exactly n bytes or fail somehow (maybe returning an error but with the available bytes, or whatever is decided) receive_lines(n): Give me exactly n lines or fail like receive_bytes()
I started implementing this in the branch svn co --username guest --password "" svn://svn.insecure.org/nmap-exp/david/nmap-readbuf I attached a script that shows how it works. nsock_readbytes and nsock_readlines (and hence nmap.receive_bytes and nmap.receive_lines) return exactly what you ask for, not more and not less. If there are not enough bytes available, the functions return an error and the bytes stay in the buffer for the next read. If there are more than enough bytes available, only the first n bytes or lines are returned, and what's left over remains in the buffer. Here's the output running against this server: $ ncat -l -k 1234 --sh-exec 'yes ABCDEFGHIJKLMNOPQRSTUVWXYZ' 1234/tcp open hotline syn-ack | readbuf: | receive_bytes(5) -> 5 "ABCDE" | receive_bytes(10) -> 10 "FGHIOJKLMN" | receive_lines(1) -> 13 "OPQRSTUVWXYZ\ | " | receive_lines(2) -> 56 "ABCDEFGHIOJKLMNOPQRSTUVWXYZ\ | ABCDEFGHIOJKLMNOPQRSTUVWXYZ\ | " | receive() -> 99 |_receive() -> 56472 "\" at the end of a line stands for "\n". The first actual network read returned 183 bytes, which was enough for receive_bytes(5), receive_bytes(10), receive_lines(1), receive_lines(2), and receive(). Compare this to what the output is in the trunk (literal strings omitted because they're too long): 1234/tcp open hotline syn-ack | readbuf: | receive_bytes(5) -> 183 | receive_bytes(10) -> 8192 | receive_lines(1) -> 58416 | receive_lines(2) -> 30408 | receive() -> 31856 |_receive() -> 34752 I think that the behavior of nsock_readbytes and nsock_readlines in my branch is just write. My question is, how should nsock_read work? nsock_read is used to mean both of these two things: 1. Keep reading until timeout or EOF, and return everything. 2. Return me some small chunk of data, up to one buffer's worth, but I don't care about the exact number. In practice, nsock_receive does (1), except that there a built-in limit of 589823 bytes, after which the read always finishes. (Look for the comment "spews and spews data" in nsock_core.c.) When people want (2), they have been using nsock_readbytes(1), which only does one recv and only returns up to a relatively small fixed-size buffer (like recv). My personal feeling is the nsock_read should work like (2). But we still want a function that works like (1) and returns a big block of data all at once. Service scan does this for example. But even service scan has no use for 589823 bytes; it seems like the caller should be able to set this limit. "As many bytes as are available, but not more than X." This is pretty close to how nsock_readbytes works now, in that it quits reading once enough bytes are available, except that you can still get more than you asked for (up to one fixed-size buffer's worth more). So I'm thinking of a function like nsock_read_upto(size_t n) Service scan would call this with n = 10240 or thereabouts. Another option is to add an argument to nsock_read to give it nsock_read_upto semantics. Current nsock_read is much like nsock_read_upto(589823). David Fifield
Attachment:
readbuf.nse
Description:
_______________________________________________ Sent through the nmap-dev mailing list http://cgi.insecure.org/mailman/listinfo/nmap-dev Archived at http://seclists.org/nmap-dev/
Current thread:
- Nsock read buffer David Fifield (Nov 15)