Re: [NSE] http-enum and http-fingerprints enhancement suggestions

[Jesper Kückelhahn <dev.kyckel () gmail com>]

On Thu, Mar 7, 2013 at 5:39 AM, David Fifield <david () bamsoftware com> wrote:

> On Sun, Feb 03, 2013 at 03:08:15PM +0100, Jesper Kückelhahn wrote:
> > The http-enum script currently displays the results in the standard
> > script output area regardless of whether the information found is
> > version related. I'd like to modify the script and fingerprint file,
> > such that it is possible to add fingerprinted web applications to the
> > 'extra info' field of the ports table. I think that this extra
> > information in the standard nmap output would make sense and would add
> > value to the service detection.
>
> I agree, it's nice if http-enum is allowed to set version fields. You
> should use your ike-fingerprints design as a model.
>
> > These modifications would require some general changes to the
> > information stored in the fingerprint file. I've listed the properties
> > I think would be needed/make sense to include:
> >
> >   - type: [version, comprehensive, intrusive]
> >   - rarity: [?]
> >   - method: [HEAD, GET, OPTION]
> >   - favicon: md5 hash
> >   - page_hash: md5 hash
> >   - probe: url
> >   - match: regex
> >   - output: string
> >   - source: [Nikto, Blindelephant, ...]
> >   - cpe: cpe:/a:...
> >
> > I've included favicon here as I think it would be a good way of
> > centralising data, and possibly joining the scripts. Also adding a
> > rarity field would allow for limiting the amount of data sent during
> > the fingerprinting. This could be based on the popularity, port
> > number, etc. I haven't figured out the best way of doing this, or if
> > it's actually a good idea. The page_hash is inspired by
> > BlindElephant[2] and [3]. I've added a source property in case
> > fingerprints were included from other sources.
>
> I don't understand the "type" of version, comprehensive, or intrusive.
> This part feels underspecified to me. How does the user interface for
> this part work? How do users choose which types of probes they run, and
> do they care? Is is possible to remove this part?


> I disagree with giving favicon any special treatment. Isn't it just a
> matter of defining a probe and having a list of match hashes?
>
> Rarity is a good idea, but it should apply only to probes, not to
> matches. Once we've done the probe, we have few enough matches that we
> can regard matching patterns against the page text as free.
>
> I don't see any fields above for vendor, version, extra info? Isn't that
> one of the main ideas?
>
> I think it would help if you could show us examples of real probes and
> matches using this system.
>
> David Fifield

Thanks for the reply and feedback.

The idea behind the 'type' option is that it would be possible to control
the amount of request the script sends. As I noted in the original post, I
think it would be great if a shortcut or scan type were to be added, that
included many of the http scripts already created. In such a scenario,
'type' would probably default to 'version', to limit the amount of traffic
send to the target(s). Maybe 'profile' would be a better naming of this
option, and the values could then be 'passive', 'aggressive' and
'intrusive'.

I'm currently working on a script that is based on this idea. It initially
makes three requests: index page, error page and favicon. If the
'type'/'profile' is 'passive', version detection is only based on these
three responses. This might detect the type of web app used, but not the
exact version. If the 'aggressive' type/profile is used, additional probes
will be requested, but only in cases where matches have been found in the
first three initial requests. 'Intrusive' type/profile would request all
the 'aggressive' probes in the database to try and find matches.

I've simplified the script a bit compared to the original post, so that
rarity and source are not included. I figured these could be added later if
needed/wanted.

I've included two samples with the current database structure and below is
a short description of the fields:
  - name: Product name
  - category: 'version', 'attack', 'info', etc (currently I've mainly
focused on 'version')
  - cpe: cpe value
  - passive: matches for the initial three requests
    - version: detected version or nil
    - request: which request to match (index, error, or favicon)
    - type: string or md5
    - match: a string or md5 sum to search for
  - aggressive: makes addition requests if a 'passive' match is found and
profile is 'aggressive'
    - version: version info
    - url: the path to request
    - type: string or md5
    - match: a string or md5 sum

A successful match would add 'name' + 'version' to the 'extra info' in the
port table.

----- Samples -----

table.insert(fingerprints, {
  name     = 'Sitecore CMS',
  category = 'version',
  cpe      = nil,

  passive = {
    {version=nil, request="error",   type="string",   match='<link
href="/sitecore/shell/applications/analytics/visitor.css.aspx"'},
    {version=nil, request="error",   type="string",   match='<link
href="/sitecore%20modules/shell/'},
    {version=nil, request="error",   type="string",   match='<meta
id=scPath content="/sitecore/content/Home/404" name=scPath>'},
  },

  aggressive = {
    {version='\\1.\\2.\\3 (rev. \\4)',
url='/sitecore/shell/sitecore.version.xml', type="string",
match='<major>([^<]*)</major>.*<minor>([^<]*)</minor>.*<build>([^<]*)</build>.*<revision>([^<]*)</revision>'},
    {version='\\1', url='/sitecore/login/default.aspx', type="string",
match='class="SystemInformationDivider">.*Sitecore.NET ([^<]*)</div>'},
    {version='\\1', url='/sitecore/login/default.aspx', type="string",
match='<hr/>.*Sitecore version ([^<]*)</div>'},
    {version='\\1', url='/sitecore/login/default.aspx', type="string",
match='class="SystemInformationDivider">.*Sitecore.NET ([^<]*)</div>'},
  }
});


table.insert(fingerprints, {
  name     = 'phpMyAdmin',
  category = 'version',
  cpe      = nil,

  passive = {
    {version=nil, request="favicon",   type="md5",
 match='d037ef2f629a22ddadcf438e6be7a325'},
    {version=nil, request="index",     type="string",
match='<title>%s*phpMyAdmin%s*</title>'},
    {version=nil, request="index",     type="string",
match='function%s*PMA_focusInput'},
  },

  aggressive = {
    {version='\\1',      url='/Documentation.txt',
 type="string", match='phpMyAdmin (%d[%.%d%d?]*) Documentation'},
    {version='\\1',      url='/Documentation.html',
type="string", match='phpMyAdmin (%d[%.%d%d?]*) - Documentation'},
    {version='\\1',      url='/README',
type="string", match='%sVersion (%d[%.%d%d?]*)'},
    {version='\\1',      url='/translators.html',
type="string", match='phpMyAdmin.(%d[%.%d%d?]*).%-.Official.translators'},

    {version='< 3.4.0',  url='/print.css',
 type="md5",    match='2a5f1cd6bac028b6aae9c1869d9fb1cb'}, --  <  3.4.0
    {version='>= 3.4.0', url='/print.css',
 type="md5",    match='816e411e29f9de3ad01258b5ec7d917a'}, --  >= 3.4.0
    {version='< 3.4.0',  url='/themes/original/img/logo_right.png',
type="md5",    match='3128fb922eaf4e2ef96e954d9684f29c'}, --  <  3.4.0
    {version='3.4.x',    url='/themes/original/img/logo_right.png',
type="md5",    match='0fc7b8442bcc810588b4070ca5435ec3'}, --     3.4.x
    {version='3.5.x',    url='/themes/original/img/logo_right.png',
type="md5",    match='990c85780329798fcd8c7a7634c39137'}, --     3.5.x
  }
});
-----------------

The existing fingerprints from 'http-fingerprints' could be included be
adding another 'category' and profile for these.

Any thoughs, suggestions, comments and/or improvements are welcomed.


- Jesper
_______________________________________________
Sent through the dev mailing list
http://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/