Re: Nmap tty and NSE

["Patrick Donnelly" <batrick.donnelly () gmail com>]

On Wed, Dec 24, 2008 at 2:00 AM, Fyodor <fyodor () insecure org> wrote:
> On Wed, Dec 24, 2008 at 01:29:37AM -0700, Patrick Donnelly wrote:
> > My feeling is there needs to be some form of unique identifier for
> > each instance of the script (even at -d1). The simplest way to do that
> > is output the thread's pointer.
>
> The thread pointer is only unique for concurrent instances, right?  I
> imagine that it may be reused in the next host group during an Nmap
> scan.

Yes it is unique for concurrent instances. It is possible that it is
reused in another scan but I don't see them getting confused.

>  I don't really have an opinion on whether a unique ID is
> worthwhile, or what the best identifier would be.  A counter would at
> least be unique during Nmap execution, though perhaps the pointer is
> more useful for debugging purposes, etc.

The pointer is more natural because a script writer has "access" to it
from Lua using coroutine.running [1]. More accurately, they may print
out the pointer using print(tostring(coroutine.running())).

> We should probably have a use in mind for it if the unique ID is
> included.  For example, some other messages can then print the unique
> ID and omit other (longer) information.

Right now NSE does not track information relevant to a script (the
filename for instance). Under some circumstances, such as when a
script thread must end (host timed out), we cannot retrieve this
information easily. Using the thread's pointer as an identifier is the
only solution that won't dramatically increase the code's complexity.

> > Finished script './scripts/showHTMLTitle.nse' (thread: 0x84b0698)
> > against 88.217.63.66.adsl.dyn.setel.com (66.63.217.88).
> >
> > Naturally, the hostname can be quite long making it very easy to
> > overflow 80 characters. The previous output I pasted didn't have any
> > hostnames for any of the targets.
>
> That one seems easy -- just remove the host name.  We don't need both
> the hostname and IP repeated on every debug line, and if you only have
> one of them, the IP is clearly the way to go.

Ok.

> It is probably worth adding the time (since Nmap started) to the
> start/finish lines.

That looks like a good idea. Which function should I be using to add the timing?

> Regarding these samples:
>
> Starting script './scripts/http-auth.nse' (thread 0x84391f8) against target 95.78.76.91.
> Finished script './scripts/http-auth.nse' (thread 0x84391f8) against target 95.78.76.91.
>
> First, I think the word "script" and "target" can probably be safely
> removed.  Also, service scripts like these will need a port number,
> like so:
>
> Starting './scripts/html-title.nse' (thread 0x843a6a8) against 95.78.76.91:80.
> Finished './scripts/http-auth.nse' (thread 0x84391f8) against 95.78.76.91:80.
>
> Perhaps we should prefix with the subsystem and time to match some of
> our other messages (particularly our trace lines).  And if the ID is
> going to be there, I'd put it at the end. For example:
>
> NSE (0.41s) starting './scripts/html-title.nse' against 95.78.76.91:80 (thread 0x843a6a8).
> NSE (0.92s) finished './scripts/http-auth.nse' against 95.78.76.91:80 (thread 0x843a6a8).

Should we add the "NSE" or "SCRIPT_ENGINE" prefix to these lines?

> A few more things to consider:
>  o The thread ID could be removed if not needed.  I don't really have
>   an opinion on whether it is worth including.  The key is to figure
>   out whether (and how) it is useful to an Nmap developer.  If you
>   can't figure out how it helps, it should be removed.  If it is very
>   useful, then it is worth keeping even if it takes us beyond 80
>   columns.

It is in my opinion perhaps the most useful piece of information. I
have often used similar debugging output (that is, added output to NSE
which printed thread pointers) when I have worked on scripts for
tracking resource allocation, when a script finishes, debug output,
etc.

>  o The finished line could tell the script return status (e.g. whether
>   it returned something, returned nothing, erred, or timed out).

That would certainly be useful. I will add it if there is demand for it.

I have attached an updated patch which prints output like so:

Running 6 script threads:
Starting './scripts/sshv1.nse' against 127.0.0.1:22 (thread 0x8541f90).
Starting './scripts/banner.nse' against 127.0.0.1:22 (thread 0x8544a98).
Starting './test.nse' against 127.0.0.1:22 (thread 0x8545a08).
Starting './scripts/ssh-hostkey.nse' against 127.0.0.1:22 (thread 0x85480b8).
Starting './scripts/banner.nse' against 127.0.0.1:3690 (thread 0x8551338).
Starting './test.nse' against 127.0.0.1:3690 (thread 0x85522b8).
Finished script against 127.0.0.1:3690 (thread 0x8551338).
Finished script against 127.0.0.1:22 (thread 0x8544a98).
Finished script against 127.0.0.1:22 (thread 0x8541f90).
Finished script against 127.0.0.1:22 (thread 0x85480b8).
SCRIPT ENGINE: target 127.0.0.1 timed out.
Finished script against 127.0.0.1:3690 (thread 0x85522b8).
SCRIPT ENGINE: target 127.0.0.1 timed out.
Finished script against 127.0.0.1:22 (thread 0x8545a08).
Completed SCRIPT ENGINE at 03:05, 2.06s elapsed

Besides adding most of your suggestions, the most notable difference
is I removed the script filename from the "Finished ..." messages
because the script thread may be in a state where we cannot know this
information (particularly, if the target host timed out). This makes
the thread pointer identifier all the more important.

[1] http://www.lua.org/manual/5.1/manual.html#pdf-coroutine.running

Cheers,

-- 
-Patrick Donnelly

"One of the lessons of history is that nothing is often a good thing
to do and always a clever thing to say."

-Will Durant

Attachment: nse_instances2.patch
Description:

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org