Ncrack initial draft specifications

[ithilgore - <ithilgore.ryu.l () gmail com>]

Ncrack initial draft specifications and thoughts
==================================

Hello nmap-dev! As you have already read from Fyodor's mail (
http://seclists.org/nmap-dev/2009/q2/0238.html ), this year's SoC will
involve
making a new network authentication cracker under the name Ncrack.

Fyodor and I had a discussion some days ago and concluded that a solid plan
is
needed before starting to code. So in this post, I will outline my thoughts
on
some aspects of a possible architecture of Ncrack.

First of all, since we should get all the existing tools' best ideas and
maybe
the most representative of them is thc-hydra, I have been studying its
source
code to see how its internals work. Below follow some notes I took. You can
skip
this part if you are not that interested in thc-hydra, but I personally
think
that getting a good grasp on one of perhaps the most competitive open source
authentication cracking tools out there, can certainly be of use and thus I
suggest that you actually do read it.


**************************** thc-hydra ***********************************

How thc-hydra works
================


Hydra is a parallelized network authentication cracker written by Van Hauser
of
tch. It is based on a modularized architecture, where each module
corresponds to
a certain protocol service. Each service module belongs to its own
independent
.c file. The user defines what kind of service he wants to attack and hydra
spawns the equivalent module a number of times equal to the number of
concurrent
processes defined (-t option).


Architecture
=========

-- General structures

 hydra_arm: represents child information (its pid_t, a pair of unix domain
 socket descriptors used for communicating with parent, the current username
&
 password to be tested etc)

 hydra_target: represents a target - a number equal to the list of IPs
defined
 in the input file (-M option) will be created:

 /-----------------------------------------------------------\

    fill_mem(servers_ptr, ifp, 0);
    sizeservers = sizeinfile; tmpptr = servers_ptr;
    for (i = 0; i < countinfile; i++) {
        hydra_targets[i] = malloc(sizeof(hydra_target));
        memset(hydra_targets[i], 0, sizeof(hydra_target));
        hydra_targets[i]->target = tmpptr;
        while (*tmpptr != 0)
            tmpptr++;
        tmpptr++;
        hydra_targets[i]->login_ptr = login_ptr;
        hydra_targets[i]->pass_ptr = pass_ptr;
hydra_targets[i]->max_use_count =
        hydra_options.max_use;
    }

 \-----------------------------------------------------------/

 login_ptr & pass_ptr are memory regions that have been filled with the
 username/password lists from the corresponding files (-L,-P,-C)


-- Parent-child communication

The communication between the process mother and the modules happens through
the
use of UNIX domain sockets (AF_UNIX). hydra_spawn_arm() creates a pair of
unnamed unix domain socket descriptors using socketpair(), forks a new child
which executes the selected service module and passes the one copy of the
descriptor as argument. Then the child writes and reads to that descriptor
to
inform the mother process of its status (e.g it failed to connect to the
service
/ it found a new password / it exited due to an error etc). The mother
process
can also write to that descriptor to pass information such as a new
username/password pair (e.g write(hydra_arms[arm_no]->sp[0], buf, buflen);)
Possible child values are:

              /* Valid Results:
               *  n - mother says to itself that child requests next
login/password pair
               *  N - child requests next login/password pair
               *  Q - child reports that it is quitting
               *  C - child reports connect error (and is quitting)
               *  E - child reports protocol error (and is quitting)
               *  f - child reports that the username does not exist
               *  F - child reports that it found a valid login/password
pair
               *        and requests next pair. Sends login/pw pair with
next msg!
               */


-- Main process loop

The mother process' main loop uses select() to go through the unix domain
socket
descriptors of its children and test if any of them has something new to
report.

/-----------------------------------------------------------\

 /* this is the big function which starts the attacking children, feeds
    * login/password pairs, etc.! */
    while (hydra_brains.finished < hydra_brains.targets && error == 0) {
    FD_ZERO(&fdreadarms);

    ...

    my_select(max_fd+1,&fdreadarms,NULL,NULL,0,200000); for (arm_no = 0;
arm_no
    < hydra_options.tasks; arm_no++) { ...

 \-----------------------------------------------------------/

Additionally it spawns new childs using hydra_spawn_arm() as needed.


-- helper functions

fill_mem(): will read the input password file and copy each
username/password to
the (already) allocated pointer passed to it.

hydra_restore_read(): will restore a session from file
hydra_restore_write():
will save current session to file for later use


Parallelization
===========

The parallelization is achieved through process forking. The user can define
the
maximum number of concurrent tasks (processes) by using the -t option.



Interesting Points
=============

Failed connection attempts
----------------------------------------

What happens if a child fails to connect to a particular server? Hydra uses
the
notion of fail_count which keeps a note of how many attempts it tried to
connect
to the particular server (it is a member of the hydra_target struct).  Every
time a child reports a 'C' or 'E' message (connect error and protocol error)
will subsequently quit and the mother will increase the fail_count counter
of
the target. If the fail_count reaches the MAXFAIL threshold (by default it
is 3)
then hydra will never try again for the particular target (it will be marked
as
'done' in global list hydra_targets). If this doesn't happen, then the
target's
hydra_arm will set the 'redo' variable to try again later.

This is an interesting point, since we usually assume that the user provides
a
list of IP addresses that have already been scanned for a particular service
(using Nmap of course!) and thus the port is open. However, firewalls using
dynamic rulesets may block us if we surpass a certain amount of connections
inside a certain time period.


Next target selection
------------------------------

Hydra selects, at each round inside the main loop, the target that the next
spawned child will attempt to attack. The algorithm for the decision is
firstly
based on choosing the target that is attacked by the smallest number of
children and secondly (in case of draw before) on the smallest fail count
(as mentioned above).


Module engine
----------------------

Hydra's module engine is simple in general. Each module gets called by a
function with name service_<service> that has the following prototype:

void service_<service>(unsigned long int ip, int sp, unsigned char options,
char
*miscptr, FILE * fp, int port)

ip -> target ip address
sp -> unix domain socket descriptor for parent communication
options -> set to OPTION_SSL if we want to use ssl
miscptr -> pointer to additional options if module requires them
fp -> file pointer to output for successfully found username/password
    reports (can be stdout or a file)
port -> defined for non-standard listening service ports

Every network operation takes place inside each module using custom network
handlers defined in hydra-mod.c


Hydra-arms
-----------------
OK that should be heads instead of arms, according to Greek mythology on
hydra :) .


**************************** thc-hydra end ********************************



Ncrack proposed architecture
======================

Ncrack will be based on the Nsock library, which is, as you all know, a
parallel
sockets library that is currently used by Nmap's service_scan engine, NSE
and
nmap_dns.cc. I have taken some notes on some aspects of Nsock at my site's
wiki
here: http://sock-raw.org/wiki/Nmap/Nsock

Nsock uses callback handlers for all main network operations (connect, read,
write). Connect(2) is done using non-blocking sockets that adds to the
speed.
nsock_loop() is used to to go through all the socket descriptors ( using
select() - optionally with a timeout ) and upon a new event(e.g new data
arriving from network) call the corresponding callback handler that has been
previously registered with each kind of operation.  Nsock's way of working
is
fundamental in designing a proper communication API between Ncrack's core
engine and the separate modules that will be service/protocol specific. The
modules will ideally need to be simple and small, while the core engine will
be
responsible for all network operations, option parsing, timing algorithms
and
communicating with the modules.


Engine - Module API
================

To keep modules as simple as possible, we could build them as state
machines.
Ideally, they should be a certain sequence of specific steps they must carry
through to make the authentication. Communication with the main engine will
be done through a generic struct that will hold all necessary information.
Example:
    struct module_data { int state, int protocol, unsigned short port,
    char *username, char *password, ... }

module_data will have to be 'connected' with one of nsock_pool's sockets.
If I am correct, struct msiod (which is then cast to nsiod) from
nsock/src/nsock_internal.h has a void *userdata member which could be used
for
this kind of message passing.

service_scan.cc is a concrete example of using nsock and some ideas can be
used from its codebase. Thus we could have the main cracking handler in
our core engine:

ncrack_crack() {

 ... variables ...

 create nsock pool
 sendSomeServiceProbes()

 nsock_loop()
 ...

}


sendSomeServiceProbes()
{

    ...
    while () {
        nsock_connect_tcp/udp(..., connect_handler)
    }

}

connect_handler()
{
    ...
    call_module(<service_name>, (struct module_data*) nsiod->userdata)
}


Thus the connection handler will invoke the generic call_module handler
and will pass to it the module_data struct (which at first will be
initialized to certain values)
Then call_module, will invoke the proper module by parsing the
<service_name>.
An example module then would be like this:

enum { READ_OP, WRITE_OP };

http_module(nsiod *io) {

    ...
    struct module_data *mdata = (struct module_data *) io->userdata
    switch (mdata->state) {
        case INIT:
            foo;
            mdata->state = NEXT;
            mdata->operation = READ_OP; // or write_op or anything
            ncrack_request_operation(io);
        case NEXT:
            bar;
            ...
        case FINI:
            sha;
        }
}


ncrack_request_operation() (from the core_engine) will register a new nsock
event based on the operation requested on the module_data struct and will
also
pass the module_data back to the event's callback handler which will
subsequently reinvoke the module with the module_data that now has the new
state. The new state will now make the module follow a different code path
accordingly and so on.

I assume, though I am not entirely certain, that all protocols can be
dissected
to use a state machine like the above. If, however, you believe that some
protocols are that quirky that the above model will overcomplicate things
instead of simplifying them, please say so.

Now let's move on to some other things.


Timing options
===========

One of the most importan features of Nmap are its timing-tuninig options.
Since many users are already accustomed with Nmap's option conventions, I
suggest that as long as the option names don't provoke confusion, that we
keep them as is. For example, take a look at the following options (from
nmap
man page)

--min-hostgroup numhosts; --max-hostgroup numhosts (Adjust parallel scan
group sizes) .

In nmap, these two options denote the number of hosts that will be scanned
in
parallel.
Ncrack will support cracking a multitude of hosts at the same time,
so having such an option with the same name will be useful (we are referring
to exactly the same thing)


--min-parallelism numprobes; --max-parallelism numprobes (Adjust probe
parallelization) .

In nmap, this denotes the total number of outstanding probes for a
hostgroup.
In ncrack, this will denote the total number of active connections.


--max-retries numtries (Specify the maximum number of port scan probe
retransmissions) .

In nmap, this specifies how many probes will hit the same port at the same
host in case we haven't received an answer yet.
In ncrack, this can specify how many retries to connect to a certain service
before marking the host service as dead (see thc-hydra failed connection
attempts above).


--scan-delay time; --max-scan-delay time (Adjust delay between probes) .

This may need a name-change but essentially, it will have the same effect.
It will limit the number of probes (as in any write(2) network operation)
per second per host.


Target Specification
===============

Target specification should also be similar to that of Nmap's. The user
should be able to either give a list of command-line IP addresses or
hostnames as well as give an input filename with the option -iL.

Implementing options like --exclude or --excludefile and CIDR notation /
whole network range specification might at first sound like an overhead,
since blindly specifying hosts might be a rare case (users are supposed
to usually specify a list of servers that have previously scanned and thus
know that have open ports on the specified service - Ncrack will have no
portscanning functionality) but nevertheless they might be of some use.

Being able to use Nmap's output automatically is another useful and
possible feature. I think parsers for nearly every Nmap's output format
are already available (OK apart from -oS I guess :) ) so that won't be
much of a problem. However, each Ncrack instance will be supposed to
scan only one particular kind of service and consequently tha only useful
parsable information will be the host IP addresses that have that
particular port marked as open.

Finally, name resolution can happen with Nmap's already existing parallel
dns resolver.


Password policy
============

thc-hydra has a separate tool called pw-inspector that parses a password
file and enforces certain policies so that passwords that do not meet
the criteria are stripped from the output result. We could make something
similar or integrate it inside Ncrack. But having it as in-built will
result in more option overhead.



All the above are just an initial rough draft. I will probably post an
update soon including more aspects. Until then, I would be glad to hear
your thoughts and feedback on the general design, especially the engine
- module API.


Cheers,
ithilgore

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org