Options to replace select in Nsock

[David Fifield <david () bamsoftware com>]

Hi,

Nsock, Nmap's parallel sockets library, uses the select function to
determine when a socket is ready to read or write and avoid blocking.
select has a built-in limit on the number of sockets it can handle,
which is FD_SETSIZE. On Linux this is 1024. Brandon Enright tracked down
a memory corruption bug to the fact that more than 1024 sockets were
being used, overflowing the fd_set buffer.

http://linux.die.net/man/2/select

There are alternatives to select that could fix this deficiency. I'm new
to all this, so I spent some time today researching options and I want
to post some links I found. A good overview of the options on different
operating systems is

http://www.kegel.com/c10k.html#strategies

poll is a function that is similar to select but it doesn't have a limit
on the number of descriptors it can follow. poll is pretty portable,
except it isn't available on Windows before Windows Vista. poll has the
same scalability problem as select, namely that you have to iterate over
the entire descriptor list to find out which ones are ready, but that
hasn't been a problem for us.

http://linux.die.net/man/2/poll

Once you get past poll it seems all the APIs are platform-specific.
There is epoll on Linux and kqueue on BSD. Windows has something called
completion ports. All of these have some level of kernel-level support
for tracking state between calls and are considered high-performance
APIs.

http://linux.die.net/man/4/epoll
http://people.freebsd.org/~jlemon/papers/kqueue.pdf
http://msdn.microsoft.com/en-us/library/aa365198(VS.85).aspx
http://technet.microsoft.com/en-us/sysinternals/bb963891.aspx

We need to do benchmarking to see if any of the platform-specific APIs
are worth implementing. The FD_SETSIZE limit is a real problem though,
and so I think it makes sense to port Nsock to use poll if possible.

David Fifield

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org