Re: [NSE] new scripts and libraries: brute library

[Patrik Karlsson <patrik () cqure net>]

On 12 aug 2010, at 04.16, David Fifield wrote:

> On Sun, Aug 08, 2010 at 05:31:36PM +0200, Patrik Karlsson wrote:
> > I've been working on some stuff during the past few weeks, and as I'm back to work tomorrow, I thought I would 
> > "offload" it to the list.
> > Some of it I have already posted but I might have made changes since, so I'm including it all again.
> > Some of the libraries unfortunately still have chunks of (ugly) undecoded data. The foremost reason for this is the 
> > lack of documentation.
> >
> > The scripts and libraries have all undergone limited testing, but as always I welcome further testing, comments and 
> > suggestions.
> > Here's a brief description of each library and script and a zip file with everything needed inside.
>
> This is great stuff. It's hard to overstate what a valuable contributor
> you've become. In fact, it's starting to overwhelm my powers of review.

Haha, thanks.

> I don't have access to all the servers these scripts work with. Because
> your code has been of such uniformly high quality, I'm going to spend
> less time looking over the code. For this batch, I'm going to take it in
> parts, starting with the brute library.

All the environments are available as 90 days trials. 
That said, I don't think it's worth for you to spend the time setting it up as you'll be testing against the same 
default environments as me.
What would be interesting is to have it tested by someone with access to non default configurations that would behave 
differently.

> A plea to other readers: When Patrik or anyone asks for help testing
> scripts, as at http://twitter.com/nevdull77/status/20637197568, please
> take the time to test. The easiest thing to do is just run the scripts
> and report the output they produced. If one crashes, the script author
> can usually diagnose the problem quickly.

I think your making a good point here David. Testing may sound complex and difficult, but what I mean by help testing 
most of the time is exactly that.
I usually develop my code against services running in virtual environments. For most cases there pretty much running a 
default configuration.
When analyzing protocols and trying to re-implement them it's easy to miss stuff. So for most cases it helps a LOT to 
just know that the code runs against anything else than my own servers.

> > o Brute framework
> > - A smallish brute framework that provides the basic iterators and logic
> > - For the moment the current scripts make use of it:
> >    x domcon-brute - see more information below
> >    x http-brute - performs password guessing against basic authentication
> >    x http-form-brute - performs form-based password guessing
> >    x informix-brute - see more information below
> >    x oracle-brute - see more information below
> >    x svn-brute - performs password guessing against subversion 
> >    x vnc-brute - see more information below
>
> I wanted to see how easy it was to write a new brute script using only
> the documentation for the brute library. I followed the example and made
> this script to talk to a server that returns "YES\n" or "NO\n".
>
> > description = ""
> > categories = {}
> >
> > require("brute")
> > require("stdnse")
> >
> > function portrule(host, port)
> >  return port.number == 4000
> > end
> >
> > local brute_driver = {
> >  new = function(self, host, port)
> >    local o = {}
> >    setmetatable(o, self)
> >    self.__index = self
> >    o.host = host
> >    o.port = port
> >    return o
> >  end,
> >  connect = function(self)
> >    self.socket = nmap.new_socket()
> >    return self.socket:connect(self.host.ip, self.port.number, "tcp")
> >  end,
> >  disconnect = function(self)
> >    return self.socket:close()
> >  end,
> >  check = function(self)
> >    return true
> >  end,
> >  login = function(self, username, password)
> >    local status, err, reply
> >    status, err = self.socket:send(username .. ":" .. password .. "\n")
> >    if not status then
> >      -- What?
> >    end
> >    status, reply = self.socket:receive_bytes(1)
> >    if not status then
> >      -- What?
> >    end
> >    if reply == "YES\n" then
> >      return true, brute.Account:new(username, password, "OPEN")
> >    else
> >      return false, brute.Error:new("login failed")
> >    end
> >  end
> > }
> >
> > function action(host, port)
> >  local status, accounts
> >  status, accounts = brute.Engine:new(brute_driver, host, port):start()
> >  if not status then
> >    return accounts
> >  else
> >    return stdnse.format_output(true, accounts)
> >  end
> > end
>
> This is a little longer than the naive brute script would be, but I feel
> it is worth it for the automatic parallelization. It found a password
> for "root" quickly.
>
> $ ./nmap --datadir . --script sample-brute 192.168.0.190 -p 4000 -d2 -Pn -n --script-args brute.firstonly=1
> PORT     STATE SERVICE        REASON
> 4000/tcp open  remoteanything syn-ack
> | sample-brute:
> |
> |   Accounts
> |     root:monkey => Login correct
> |   Statistics
> |_    Perfomed 25 guesses in 1 seconds, average tps: 25
>
> What do you do when there's a socket error in the login method? Is there
> a way to report it, or perhaps ask the brute library to retry the
> credentials?

The login function may respond with either:
true, brute.Account object - if it finds a valid credential
false, brute.Error object - if it fails

So if a socket error occurs we need to report an error back to the engine.
If the retry attribute is set using the setRetry on the error object the engine will retry the credential.
When max_retries (default 3, can be modified with argument) is reached, the engine aborts.

A driver can also set the abort attribute in order to signal the engine to stop guessing.
This is suitable for example in the case of VNC where the IP is blocked from further guessing attempts after a number 
of incorrect passwords.

> A nice thing would be the ability to provide feedback to
> automatically reduce the number of threads, like the congestion control
> in the port scanning engine.

Yes, I'll put that on the todo list. I had some code that would run every X seconds and check how many attempts each 
thread had made.
When it noted threads not making any guesses it would reduce the number of running threads.

> I noticed that the script can say
> "Re-trying"; what makes this happen and how can it be controlled? It
> should be documented.

That occurs when the driver reports a retry back to the engine.

> Please also include documentation for the account states "OPEN",
> "LOCKED", and "DISABLED".

I've documented this in the brute library as:
An account have the following states
- OPEN - Login was successful
- LOCKED - The account was locked
- DISABLED - The account was disabled

I realize that there are more cases which need to be added in the future.
Also the LOCKED state should probably be divided into LOCKED_CORRECT
and LOCKED_INCORRECT or something.

> I notice that there is a brute.emptypass script argument. I think that
> this is better controlled with a password list. Our default list already
> includes the empty password.

Your probably right, I included it as a "rule" mostly so it can be disabled by scripts for services that are known not 
to support empty passwords.

> There are other script arguments that I think could be made unnecessary
> with a different interface. Instead of brute.emptypass,
> brute.useraspass, brute.unique, and brute.passonly, what do you think of
> allowing the user to provide his own username and password iterators? Or
> course the defaults would be unpwdb.usernames and unpwdb.passwords. You
> could even keep the script arguments as a convenience. I'm picturing an
> interface like this:
>
>       engine = brute.Engine:new(brute_driver, host, port)
>       engine.usernames = my_username_iterator
>
> Things like brute.useraspass and brute.unique could be nicely
> implemented as wrapper iterators, rather than special-purpose code in
> the library. I envision that the brute library would have the most
> common iterators included in it. Someone wanting a unique list of
> usernames, for example, could do
>
>       engine.usernames = brute.unique(my_username_iterator)

This is probably the way to go. My colleague Martin Swende told me so a while back, I probably should have listened to 
him :)
Anyway, I'll look into that as well.

> (Or the library could do that transparently when the brute.unique script
> argument is given.) The reason for all this is that there may be
> specialized passwords, for example, that a service wants to add, along
> the lines of brute.useraspass.

There are many cases for this like default passwords for databases or like in the case of Oracle default credentials:
sys/change_on_install, system/manager, scott/tiger etc.

> The vnc-brute script might want to do
> this:
>
>       function extra_passwords(extras)
>               local count = 0
>               local status, passwords = unpwdb.passwords()
>               if not status then
>                       return status, passwords
>               end
>               return true, function(cmd)
>                       if cmd == "reset" then
>                               count = 0
>                       end
>                       if count < #extras then
>                               count = count + 1
>                               return extras[count]
>                       end
>                       return passwords(cmd)
>               end
>       end
>       engine = brute.Engine:new(brute_driver, host, port)
>       engine.passwords = function() return true, extra_passwords({"vnc", "VNC"}) end
>
> Other iterators could do things like changing the case of letters,
> appending "1", anything. Putting the knowledge in external iterators
> means the brute library doesn't have to know about everything that's
> possible.

Good point.

> I notice that by default, the library checks every password for one
> user, then every password for the next, and so on. This can be
> controlled through the brute.mode argument. I think that putting
> passwords in the outer loop is a better default. What do people think
> about this?

I usually run very small password lists to avoid the risk of locking accounts out.
This usually means that a round of guessing is rather quick.
The default simply reflects what I'm used to, but I don't mind changing it.

> Another default is to keep trying credentials after a successful set is
> found, in an attempt to find them all. Will this last until the unpwdb
> time limit expires? What does everyone prefer for a default for this?
>
> Related to the previous paragraph, if there's going to be a debug
> message for every attempted credential, then there should definitely be
> one for a successful login.

There now is one, and there previously was, must have been lost somewhere along the way.

> I don't understand the purpose of the check method. For many protocols
> its just going to return true. All it does is stop the engine at the
> very beginning if the function returns false; couldn't that just as
> easily be done in the action function, and simplify brute's interface?

Hmm, sure, that's probably a better way, I guess I over thought that function a bit.

> Since you've made it so easy to control the number of threads with
> brute.threads, this is a good opportunity to measure the effect or
> parallelism. I tried vnc-brute against the LAN with different script
> counts. In this case the difference was slight but maybe other protocols
> or network scenarios will do better.

To be honest, I've seen very little difference myself so far as most my machines are running heavy on CPU with a single 
thread but then again my setup is far from optimal.

> brute.threads=1
> |_    Perfomed 2498 guesses in 8 seconds, average tps: 312
> brute.threads=10
> |_    Perfomed 2498 guesses in 7 seconds, average tps: 356
>
> I want you to commit this library as it is, and make further changes
> under revision control. I think this is close enough to what is wanted
> that we shouldn't fuss with it too much before committing it.

Done, the library with the changes to documentation and debugging is commited as r19750.
The library now also defaults to guessing the first password for each user i.e. it has the password iterator as outer 
loop.

> David Fifield

//Patrik

--
Patrik Karlsson
http://www.cqure.net
http://www.twitter.com/nevdull77





_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/