Re: CouchDB scripts

[David Fifield <david () bamsoftware com>]

On Sun, Jan 31, 2010 at 09:21:28PM +0100, Martin Holst Swende wrote:
> David Fifield wrote:
> > On Wed, Jan 27, 2010 at 09:17:41PM +0100, Martin Holst Swende wrote:
> >   
> > > A Couchdb library and scripts is now finished(at least in alpha). It
> > > consists of :
> > > * json.lua - library for parsing json
> > > * couchdb-databases.nse - script which get database info
> > > * couchdb-get-stats.nse - script which get some runtime statistics from
> > > the database.
> > >
> > > Sample output is below (the bottom one may be a bit verbose - perhaps I
> > > should compact that a bit.
> > >
> > > The scripts can be found at the same place (or via hg pull ; hg update).
> > > For future reference : would you rather I send files via email ?
> > >     
> >
> > Thanks. These scripts will be good ones to have. I tested it against
> > CouchDB installed fresh from MacPorts. The first time running after
> > starting the server I see
> >
> > PORT     STATE SERVICE
> > 5984/tcp open  unknown
> > |_couchdb-get-stats:
> > |_couchdb-databases:
> >
> > The second and every following time I run the script, I get
> > [snip]
> >   
> No, I have no idea. I don't suppose you kept the -d3 log showing the
> traffic ?

I attached the debug output and packet captures. It looks like the first
time, the server is returning the empty object {}, and it returns real
data after that. I think your script is handling is properly, but I
don't know why the server would do that. Maybe a bug?

> > I would like json.qtrim to handle backslash escapes, and signal an error
> > if it doesn't get a valid quoted string. It looks like fromJson can
> > silently return bad data in the case of an error. I added these to your
> > test cases:
> >
> >         '() test ()',
> >         '1 / 2',
> >         '"gaz\\"onk"',
> >         '{foo:"gaz\\"onk", pi:3.14159,hello:{ wo:"rld"}}',
> >  
> > They decode and flatten to, respectively,
> >
> >     () test (),
> >     1 / 2,
> >     gaz\,
> >     {hello={wo=rld,},pi=3.14159,foo=gaz\,}
> >
> > I would expect an error in the first two cases and
> >
> >     gaz"onk
> >     {hello={wo=rld,},pi=3.14159,foo=gaz"onk,}
> >
> > for the last two. 
>
> This is where things start getting tricky. Regarding keys, which the
> qtrim is used for, we have some rules : they must be valid javascript
> object attribute names. This is valid:
> a={"fo\"o":"bar",vax:"boork"};
>
> So, I will implement unescape for keys, but they do not have to be
> quoted in the first place (like a.vax), so it should not give errors in
> those cases.
>
> For values, however, it is more tricky, since they can be pretty much
> anything :
>
> a={a:1/2, b:alert, c:alert(1), d:{}+1, e:""|''|""}
>
> a.a => 0.5 (number)
> a.b => the window.alert function (native function)
> a.c => the return of alert, (undefined)
> a.d => "[object Object]1" (string)
> a.e => Three empty strings, OR:ed into 0 (number)

I think you are making this more complicated than it really is. We're
not parsing JavaScript, we're parsing JSON. RFC 4627, section 2, says

        A JSON value MUST be an object, array, number, or string, or one
        of the following three literal names:
                false null true

Things like 1/2 and alert(1) are JavaScript, not JSON. In JavaScript
they may return a value or whatever, but they're a syntax error for us.

For strings, the only escapes you have to handle are \", \\, \/, \b, \f,
\n, \n, \r, \t, and \uXXXX. (I'm looking at section 2.5 of RFC 4627.)
For the \uXXXX escape, I guess you should ideally collapse the
character into its UTF-8 encoding. We only have 8-bit strings in NSE so
Unicode text has to stay encoded.

And also according to the RFC, all strings have to be quoted. The a.vax
is Lua or JavaScript notation, not JSON.

> > I like the flatten functionality, but you should make
> > it return JSON so that the library can be used to encode as well as
> > decode. If you just want a quick look at a Lua table, use
> > nsedebug.tostr.
> It was just a quick look I wanted, so far there's no need for me to
> encode json. Perhaps save that for later? Thanks for the input.

It's just a small modification to what you have already written.
Something like:

function flatten(obj)
  if obj == nil then
    return "null"
  elseif obj == false then
    return "false"
  elseif obj == true then
    return "true"
  elseif type(obj) == "number" then
    return string.format("%g", obj)
  elseif type(obj) == "string" then
    return string_escape(obj)
  elseif type(obj) == "table" then
    local k, v, elems
    elems = {}
    if #obj > 0 then
      -- Array
      for _, v in ipairs(obj) do
        elems[#elems + 1] = flatten(v)
      end
      return "[" .. table.concat(elems, ", ") .. "]"
    else
      -- Object
      for k, v in pairs(obj) do
        elems[#elems + 1] = string_escape(k) .. ": " .. flatten(v)
      end
      return "{" .. table.concat(elems, ", ") .. "}"
    end
  else
    error("Unknown data type in flatten")
  end
end

David Fifield
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/