Re: Timing race cars with a sundial (-sV match performance)

[Brandon Enright <bmenrigh () ucsd edu>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 8 May 2009 17:27:58 -0700
Fyodor <fyodor () insecure org> wrote:

> On Fri, May 08, 2009 at 10:07:03PM +0000, Brandon Enright wrote:
> > A few weeks ago I embarked on a (mostly) futile attempt to measure
> > the performance of applying a PCRE match expression in our
>
> Thanks for sending the results, even if they weren't all you hoped.

Actually the results were better than I hoped.  I was under the mistake
impression that a lot of our more inefficient match lines actually
translated to slow matching.  Instead of finding out that we really
need to fix hundreds of matches we learned that we probably only need
to fix ~5 :-)

> > As for optimizing these matches, for the LPD match we really need to
> > add an anchor.  I did some checking and all UCSD hosts that match
> > the service can safely add '^' and still match.
>
> Sounds like a good idea.
>
> > The '^.*' usage in the ser2net match is counter-productive.  I don't
> > have any matching services so I don't know what can be done to
> > improve on '^.*' besides just removing it.
>
> Removing the "^.*" sounds like the way to go, as it seems completely
> pointless to me.

Will do.  It won't actually make the match any faster but it will at
least make study go faster.

> > The Xylan PizzaSwitch telnetd match is pretty zealous in its use
> > of .* early in the match.  Telnet services often match the start and
> > then print a large amount of data (banners, abuse warnings, etc).
> > '.*' is consuming all of that data on all telnet services and then
> > backtracking a byte at a time.  We should make the '.*' lazy by
> > changing it to ".*?". Even better would be to add a few more
> > matching bytes to match the telnet control bytes before using '.*'
> > but we may not have enough data to do this.
>
> I'll forward you some Xylan PizzaSwitch signatures in a few minutes in
> case it helps to have the extra data.

(I just got them, thanks!)

> > The Cyrus Murder matches look like a '^' can be added.  The protocol
> > looks like IMAP and it is safe to anchor the \* in IMAP with '^'.
> > UCSD doesn't have any Cyrus Murder installs for me to test.  I'd
> > suggest we add the anchor and then wait for new submissions if it
> > doesn't match.
>
> Sounds reasonable.

Another user (thanks!) shared with me their server output which looked
something like:

* AUTH "GSSAPI"
* PARTIAL-UPDATE
* OK MUPDATE "hostname" "Cyrus Murder" "v2.3.13" "(master)"

This means that although Cyrus Murder is in the spirit of IMAP it
isn't exact and a ^ won't work.

We can still make a failing match /much/ faster though.  We need to
tell PCRE that instead of skipping bytes that couldn't possibly match
(aka .*) the format is "\* something\r\n".

I propose:

m|^(\* [^\r\n]+\r\n)*?\* OK...

PCRE should be able to fail quickly rather than trying to apply the
match to every byte offset in the service.

> > I'm happy to submit a patch that does all of the above if it sounds
> > reasonable.
>
> Yeah, please do create such a patch and check it in.

I'll leave the Murder changes above out unless you think it is okay to
put such a regex in the probes file.

> > We might also think about adding Nmap internal performance
> > statistics logging to Nmap proper similar to my addition of
> > log_write(LOG_STATS, ...) in this branch.  I feel like sometimes
> > using
> > - -d3 or more is too much data when all you want to do is measure
> > performance stats.
>
> Have written up anywhere how this feature works?  We do try to avoid
> filling Nmap with too much debugging code, but sometimes it is
> definitely worthwhile.

Okay I'll come up with a proposal.

Brandon

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)

iEYEARECAAYFAkoE0ocACgkQqaGPzAsl94IHqwCeLnOEa5JDu5gIkhGBB+USofav
6ykAn0jp0OU1hhdsVwvuoxkzMtoCCotV
=OlkQ
-----END PGP SIGNATURE-----

_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://SecLists.Org