Re: ncat-test.py?

[Jacek Wielemborek <d33tah () gmail com>]

01/01/2014 19:15:11 Jacek Wielemborek <d33tah () gmail com>:
> Hi,
>
> On the CCC I was talking with David about rewriting Ncat test engine so that
> the tests would run concurrently, speeding up the testing process. I
> suggested a rewrite to Python and heard that it's a good idea. I already
> created the branch and I wanted to ask you guys if you have some special
> expectations for the testing engine, apart from concurrency. For one, I
> have an idea to make all tests have an indentifier (and/or a group) to
> allow stuff like:
>
> ncat-test.py --run-tests=max-conns-0-ko-exec,http-proxy-missing-digest
>
> What do you think?
>
> Yours,
> Jacek Wielemborek

Hi again,

I did a bit of experimentation and wrote a first prototype of ncat-test.py. 
It's currently just a proof-of-concept and has some kludges, but I wanted to 
ask you guys if it's generally the direction we'd like to go for. 

In case you wanted to take a look at the code before reading the rest of the 
e-mail, it's in the SVN (nmap-exp/d33tah/ncat-test-py). You can also see it on 
Github as I keep these two repositories in sync - the core (and currently the 
only) file can be found here:

https://github.com/d33tah/ncat-test-py/blob/master/ncat/test/ncat-test.py

My Python demo has currently two tests implemented - one being the first one 
from ncat-test.pl, which spawns "ncat -l" and tries to connect to it over IPv4 
and IPv6, also using Ncat. The second test tests UDP IPv6 in pretty much the 
same way. These two tests run concurrently. Tested under Python 2.6.6 (which 
AFAIK can still be found on latest CentOS Linux), 2.7.5 and 3.3.2 under Fedora 
19 (Linux). It seems to work under FreeBSD as well.

There's no Windows/Cygwin support at the moment - do_read function would have 
to be rewritten to support non-blocking operations there and apparently Cygwin 
doesn't support Posix version well (switching subprocess's stdout to a non-
blocking mode prevented me from terminating the process later). Windows 
support is currently not a priority for me, please let me know if you consider 
this a show-stopper.

As for the code, I wanted to make the tests look similar to ncat-test.pl. The 
general idea is that we create separate functions for each test, adding the 
@ncat_test generator which is supposed to register the test to the TESTS 
global variable and add some metadata, such as the test name and whether it's 
expected to fail.

Then, once run_tests(), the main function, is run, we spawn a fixed number of 
worker threads that share a queue. On the main thread, we distribute the tests 
on the queue and the threads will receive them. This has the nice advantage 
that we can configure the maximum number of tests that can be run 
simultaneously. Since the tests_worker() and handle_test(), it will be trivial 
to convert this code to a single-threaded mode, which could prove useful while 
debugging.

My main question is whether you like this approach. Before I'll start porting 
all the test cases to this new program, I'd prefer to find out whether there's 
anything potentially fundamental that should be changed. For example, folks 
from the #python IRC channel frowned upon my ScopedPopen class and suggested 
that I should use Twisted instead. I don't know Twisted very well and while it 
would probably solve some problems related to non-blocking reading, it would 
also require basically a complete rewrite of this program. This is why I'd 
like to know your opinion on this.

For the ones that would like to help me a bit and test it, currently you just 
need ncat somewhere in your $PATH and grab ncat-test.py and run it:

wget https://svn.nmap.org/nmap-exp/d33tah/ncat-test-py/ncat/test/ncat-test.py
python ncat-test.py

Here's the expected output:

$ python ncat-test.py
SUCC:   Server default listen address --udp IPV6
SUCC:   Server default listen address and port IPv4
2 tests ran, 100.00% success rate (2 SUCC, 0 FAIL, 0 UNEXSUCC, 0 XFAIL)

So, does it work for you? Do you have any comments on your mind? I'd be happy 
to hear any feedback :)

Yours,
Jacek Wielemborek

Attachment: signature.asc
Description: This is a digitally signed message part.

_______________________________________________
Sent through the dev mailing list
http://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/