Re: [NSE] [patch] Big changes to http-enum.nse

[Ron <ron () skullsecurity net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, 18 Oct 2010 13:22:42 -0600 David Fifield <david () bamsoftware com> wrote:
> On Mon, Oct 18, 2010 at 06:25:15AM +0200, Patrik Karlsson wrote:
> > On 17 okt 2010, at 22.55, Ron wrote:
> > > All right, I'm attaching my newest patch (and the
> > > http-fingerprints.lua file separately, in case people just want
> > > to check that out). I think it addresses all the ideas we've
> > > thrown around so far in a pretty nice way. The configuration file
> > > is now a .lua and basically builds a fairly flexible table. There
> > > is a bunch of validation in the script to ensure the person
> > > didn't miss a required field or use an incorrect variable type,
> > > too. 
> > >
> > > I need to go over the fingerprints file and do some cleanup, but
> > > the actual functionality is here now. 
> > >
> > > Comments would be great! 
> >
> > While being a lot more flexible, wouldn't the new format still
> > require four match lines for eg. Outlook Web Access in the
> > following examples?
> >
> > table.insert(fingerprints, { path='/mail/', verb='GET',
> > matches={ {match='*owa*', output='Outlook Web Access'} }})
> > table.insert(fingerprints, { path='/webmail/', verb='GET',
> > matches={ {match='*owa*', output='Outlook Web Access'} }})
> > table.insert(fingerprints, { path='/', verb='GET',
> > matches={ {match='*owa*', output='Outlook Web Access'} }})
> > table.insert(fingerprints, { path='/owa/', verb='GET',
> > matches={ {match='*owa*', output='Outlook Web Access'} }})
> >
> > I was thinking more along the lines:
> >
> > Probe { path="/mail/", verb="GET"  }
> > Probe { path="/webmail/", verb="GET"  }
> > Probe { path="/", verb="GET"  }
> > Probe { path="/owa/", verb="GET"  }
> >
> > match { status="200", body="*owa*", desc="Outlook Web Access" }
> >
> > I might be missing something that makes the choice of splitting the
> > Probe and match like this a very bad idea?
>
> I think that will be too expansive, if you mean that every match will
> be matched against every probe. Like this path for example:
> /archive/flash:home/html/images/Cisco_logo.gif
> It's not a good idea to grep the GIF file for "*owa*".
>
> In nmap-service-probes we have a system where one probe has multiple
> matches. If you want the same match for two different probes, you
> either duplicate the match line, or (better) declare a fallback probe
> (http://nmap.org/book/vscan-technique.html#vscan-cheats-and-fallbacks).
>
> What Ron is describing is a system where multiple paths can share
> multiple matches in common. It's as if each path has a fallback to all
> the others. If it's truly meaningful for a match at different paths to
> mean different things (different versions for example), then the
> matches will have to be duplicated in two different lines.
>
> A benefit of having the database file in Lua is that you can
> procedurally generate a bunch of probes:
> KNOWN_VERSIONS = {"0.99", "1.00", "1.00a", "1.01", "1.999"}
> for _, version in ipairs(KNOWN_VERSIONS)
>       Probe { patch="/app-" .. version .. "/", ... }
> end
>
> As for this format:
>
> table.insert(fingerprints, {
>  path='/phpmyadmin/',
>  verb='GET'
>  matches={
>   {match='PhpMyAdmin (.*)', output='Found PhpMyAdmin version \1'},
>   {output='Found PhpMyAdmin, unknown version'}
>  },
> )
>
> I think that the method ("verb") and path should be combined. Like
> this:
>
> table.insert(fingerprints, {
>  probes={{verb='GET', path='/phpmyadmin/'}},
>  matches={
>   {match='PhpMyAdmin (.*)', output='Found PhpMyAdmin version \1'},
>   {output='Found PhpMyAdmin, unknown version'}
>  },
> )
>
> Multiple paths would be like this:
>
> table.insert(fingerprints, {
>  probes={{verb='GET', path='/phpmyadmin/'},
>          {verb='GET', path='/PhpMyAdmin/'}},
>  matches={
>   {match='PhpMyAdmin (.*)', output='Found PhpMyAdmin version \1'},
>   {output='Found PhpMyAdmin, unknown version'}
>  },
> )
>
> But the default method would be GET, so you could do simply
>
> table.insert(fingerprints, {
>  probes={'/phpmyadmin/', '/PhpMyAdmin/'},
>  matches={
>   {match='PhpMyAdmin (.*)', output='Found PhpMyAdmin version \1'},
>   {output='Found PhpMyAdmin, unknown version'}
>  },
> )
>
> The idea is to flexibly support other information that we want to vary
> along with the path. (Like not forcing all paths to have the same
> method.)
>
> table.insert(fingerprints, {
>  probes={'/phpmyadmin/', '/PhpMyAdmin/',
>          {verb='POST', '/phpmyadmin/whatever',
> postdata={action='list'}}}, matches={
>   {match='PhpMyAdmin (.*)', output='Found PhpMyAdmin version \1'},
>   {output='Found PhpMyAdmin, unknown version'}
>  },
> )
>
> This could also be used to add a rarity to each path à la
> --version-intensity.
>
> David Fifield

All right, I changed it to that format and committed it to my new branch:
svn co --username=guest --password='' svn://svn.insecure.org/nmap-exp/ron/nmap-http

I changed the individual probes to have tables, even though the default recognizes strings:

table.insert(fingerprints, { probes={{path='/analog/', verb='GET'}}, matches={ {match='', output='Potentially 
interesting folder'} }})

table.insert(fingerprints, { probes={{path='/anthill/', verb='GET'}}, matches={ {match='', output='Potentially 
interesting folder'} }})

It's a little ugly, but I'm really liking how the cleaned up probes look:
table.insert(fingerprints, {
  probes={
    {path='/images/', verb='GET'},
    {path='/icons/', verb='GET'}
  },
  matches={
    {match='<title>Index of .*(Apache.*) Server at', output='Image directory w/ listing on \'\\1\''},
    {match='<title>Index of', output='Image directory w/ directory listing'},
    {match='',                output='Image directory'}
  }
})

Ron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (GNU/Linux)

iEYEARECAAYFAky9Fg0ACgkQ2t2zxlt4g/RaSQCeJ3Qwyhcs7CYtH5xYOJxpUVAy
bpYAoKeQGDnBUFaZeSGJtm/f2zyNQvjX
=8gQd
-----END PGP SIGNATURE-----
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/