Re: Nmap 6.45 Informal Release - citrixxml.lua accidental tarpit

[Tom Sellers <nmap () fadedcode net>]

On 4/12/2014 3:54 PM, Fyodor wrote:
> Hi Folks!  Late last night we posted Nmap version 6.45 to the web site.  It
> includes Patrik's excellent ssl-heartbleed script for detecting vulnerable
> SSL servers (http://nmap.org/nsedoc/scripts/ssl-heartbleed.html) and also
> Rob Nicholls' super quick update of our Windows OpenSSL binaries to help
> keep Nmap users safe from the same issue.  We never shipped vulnerable
> OpenSSL libraries with our Linux or Mac packages, and our new 6.45 Windows
> packages are now linked to a secure version (1.0.1g).
>
> This release also includes tons of other major improvements we made over
> the last 8 months since the 6.40 release.  Some of the improvements can be
> found in the raw-format CHANGELOG (http://nmap.org/changelog.html) and I'm
> working on cleaned up release notes now.
>
> Please give it a try and let me know if you find any problems.  If all
> seems well, I'll announce the release more prominently early next week.
>
> Cheers,
> Fyodor

I've been meaning to submit this for a couple weeks, I am sorry if it is
coming in a bit late for the new release.

There is an issue with the parsing XML from Citrix servers that can cause
the CPU of the scanning machine hit 100% for 60+ minutes during a character
substitution loop.  In my case this was due to the XML response containing
nearly 8,000 encoded space characters as part of application names in a response
that was nearly 87 MBs.  It is a large environment and the icons are encoded
in the XML.

The behavior can be tested using:

  nmap -sSC -vv --script=citrix-enum-apps-xml  -p80 <target>


In citrixxml.lua, the relevant function is 'decode_xml_document' and looks
like this:


  local newstr = xmldata

  for m in xmldata:gmatch("(&#%d+;)") do
    hexval = m:match("(%d+)")

    if ( hexval ) then
      newstr = xmldata:gsub(m, string.char(hexval))
    end
  end



So, it loops over the response and handles one character at a time,
performing a global substitution of the character and placing it in
'newstr'. In my case it performs this global substitution ~8,000 times
against the ~87 MB of data in the original variable despite the nearly
100% of the replaced characters being the same, a space.



I have changed the logic so that the loop works with the new copy of the
data, performs the global substitution, then loops again against the updated
data looking for a new encoded character to replace.

  local newstr = xmldata
  local escaped_val

  while string.match(newstr, "(&#%d+;)" ) do
    escaped_val = string.match(newstr, "(&#%d+;)")
    hexval = escaped_val:match("(%d+)")

    if ( hexval ) then
      newstr, _ = newstr:gsub(escaped_val, string.char(hexval))
    end

  end


The resulting times from my scan this morning were:

pre patch:      12,565.82 seconds   or 209 minutes  or 3.4 hours
post patch:         48.32 seconds

Also of note is that '--host-timeout' does **NOT** cause the process to stop
when the threshold is reached.  I have not had time to look into that.


I have attached the patch for review.


Thanks much,

Tom

Attachment: citrixxml_substitution.patch
Description:

_______________________________________________
Sent through the dev mailing list
http://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/