Re: [NSE] A Lua implementation of NSE

["Patrick Donnelly" <batrick.donnelly () gmail com>]

Hi Ron,

On Sat, Jan 17, 2009 at 9:09 PM, Ron <ron () skullsecurity net> wrote:
> > These are the following ways a thread may yield and whether its host
> > will still be charged time.
> > [...]
> > o A thread blocked on a mutex or condition variable (see nse.condvar)
> > will not be charged time.
> This may be fixed in other ways (such as your push/pop_handler)
> function, but one of the things that saved a lot of trouble when me and
> Brandon were testing was the fact that a thread that was stuck on a
> mutex (that is, waiting on a mutex that would never be released) would
> eventually time out, and the execution would end. Am I correct in
> thinking that, if you pushed that change, deadlocked scripts would never
> finish?

It depends, but usually yes. So, we need some method of cycling
through mutexes. In particular, we need to check if the thread locked
on the mutex is dead (that is, either finished executing or threw an
error). We don't necessarily have to look for deadlock because a
script with a locked mutex will eventually timeout; however, waiting
until a target times out so a dead script ends is most certainly
unacceptable.

I want to make a more robust solution for scripts that are unable to
progress, instead of a quick, possibly inflexible, fix. I'll post here
once I've got a solution.

> I think it's good if there's some way for scripts stuck in a mutex to
> get out, eventually, if there isn't one.
>
> The fact that your push/pop_handler would let me handle an error
> condition gracefully means that I can properly release my mutexes if
> something bad happens, so that deadlock situation should never happen,
> but there's always that chance.

Right. NSE should take measures to eventually kill deadlocked threads.

> Here's the situation: I have two scripts (smb-brute and smb-pwdump).
> smb-brute will bruteforce accounts and find weak passwords. smb-pwdump
> will use those accounts to log in and dump the password hashes for all
> users on that system.
>
> And what I'd like to do: I'd like to feed those hashes, discovered by
> smb-pwdump, back into smb-brute for other servers (at least, other
> servers in the current hostgroup). I don't want the scan against any
> server to end until every password in the dictionary plus every password
> discovered from other systems has been tried. If one of the discovered
> passwords works against a system, every system should be attempted with
> those passwords until we stop gaining ground. Does that make sense?
>
> So basically, the execution of all scans against the current hostgroup
> are dependent on each other.
>
> Is that something that can currently be done?

My first instinct is to suggest to use runlevels but this won't work
since there is a circular dependency. In the current version of NSE, I
don't believe there is a way to do that.

If you use a condition variable (nse.condvar), then it may be possible
to coordinate this. What you want to do is have your scripts loop on
the condition variable constantly waiting for more acc/pwds to try.
Here's some Lua code to outline this:


local condvar = nse.condvar("my unique shared ID for the two scripts");

while new_passwords_to_try do
  -- test passwords against host
  if found_passwords then
    -- add the passwords the global shared table
    condvar("broadcast");
  end
  condvar("wait");
end


There are some problems with this code though. How do you know, after
all scripts are waiting on the condition variable, when there is
nothing left to try? The answer is you should make all the scripts
"aware" of each other in the registry outside the action function. You
need to add, before waiting on the condition variable, a check if
another script is set to run in that loop. Essentially:


-- A table of scripts (threads) that will eventually run
nmap.registry.password_scripts = nmap.registry.password_scripts or {};
nmap.registry.password_scripts[coroutine.running()] = true;

local function script_will_run()
  for co, will_run in pairs(nmap.registry.password_scripts) do
    if will_run then return true end
  end
  return false;
end

local function set_all_will_run()
  local ps = nmap.registry.password_scripts;
  for co in pairs(ps) do
    ps[co] = true
  end
end

function action (host, port)
  -- set up
  local condvar = nse.condvar(nmap.registry.password_scripts); --
password_scripts table
  while new_passwords_to_try do
    -- test passwords against host
    if found_passwords then
      -- add the passwords the global shared table
      condvar("broadcast");
      set_all_will_run(); -- All scripts are set to run
    elseif not script_will_run() then -- no other scripts set to run?
      -- new_passwords_to_try is false
      condvar("broadcast"); -- wake them all up so they can exit this while loop
      break;
    end
    nmap.registry.password_scripts[coroutine.running()] = false; -- no
new data to use right now
    condvar("wait");
  end
end

This algorithm is obviously a little complicated, but, the basic idea
is to constantly wake up any scripts when any new data is available.
If all scripts are "done", in that they have all used the latest data
available, then you wake them all up so they can go on doing other
work.

The variable new_passwords_to_try will be a "local" determination, in
that, you may have some index, n, in an array that the thread has used
all passwords from 1-n. If n is the max value in the array, then there
are no new passwords to try.

Obviously the condition variable dependency requires that condition
variables are added (mutexes aren't useful for this problem) and the
coroutine change where the loading of the script itself, the execution
of the host/portrule, and the running of the action function is all
done in the same thread. In the current version of NSE, the hostrule
and script's globals (action, rule functions, etc.) are loaded through
main thread (coroutine.running() will return nil). In the new
implementation, for most people, this means that coroutine.running()
will work anywhere.

> I'm not opposed to combining those two scripts (smb-brute and
> smb-pwdump), since they're both password-stealers, but that isn't really
> the issue. The issue is more with how different parallel instances of
> the same script communicates.

That shouldn't be necessary but having inter-script dependencies may
not be favorable. Generally, if you feel a person running your scripts
would want to exclude one of the two scripts, because one is
"invasive" (category), then keeping them separate is necessary.
Otherwise, I would merge them and make use of the nse.new_thread
facility to divide up the tasks (that is its purpose after all).

Cheers,

-- 
-Patrick Donnelly

"One of the lessons of history is that nothing is often a good thing
to do and always a clever thing to say."

-Will Durant

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org