Re: [RFC] Vulnerability library proposal

[Djalal Harouni <tixxdz () opendz org>]

On Sun, Aug 07, 2011 at 11:10:45AM +0100, Rob Nicholls wrote:
> Hi Djalal,
Hi Rob,

> Overall it all looks very promising :)
>
> Is it possible we can parse the IDS values to automatically create the
> references for popular IDs (e.g. CVE, OSVDB)? Otherwise you're duplicating
> holding 'CVE-2010-4344' and
> 'http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-4344' (this also
Yes we can achieve this, we'll just creat a table of popular IDs and try
to map them to their references.

> ensures that people adding an OSVDB such as 69860 won't need to remember to
> also add the corresponding URL as a reference, which appears to have been
> missed in one of your examples).
Right, it's missing.

> Then references can be used to hold links to other third party references,
> such as posts on Bugtraq, Full Disclosure or relevant blog entries (e.g.
> blogs.msdn.com), where an ID might not (easily) exist. It also makes it
> easier to update the references (i.e. we won't have to trawl through
> hundreds of scripts) if one of the big names changes the URL used to view
> IDs (although I suspect they'd automatically redirects users to the new
> location anyway).
I think that we should also put links of the popular references.

> I must admit I'm not entirely sure about making IDS a mandatory value. What
> if someone puts together a check for something when a CVE doesn't exist?
> Sure, Apache seems to always have a CVE against every security issue it
> fixes; but less popular software might not have such references. I know we
> can create our own references, which could be a way around it though.
You have a point here, and you have already suggested a solution.

The problem is we can have different scripts that will try to check the
same vulnerability (e.g. osvdb-vuln.nse check and search the OSVDB
service for different vulnerabilities and  ftp-vulns-osvdbXXXXX.nse that
will check for the X FTP vulnerability ...). In this case having two
tables or more in the registry that reference the same vulnerability
that affects the same host is not optimal (think about a lot of hosts).
The other case is we are saving these vulnerabilities for post-processing
scripts and it would be much better if we can just report only a one entry
for each host with the correct state, and avoid reporting multiple
vulnerability entries that can have the _same_ state.

And the IDs will also help the internal lookup mechanism, so we do not
traverse all the hosts to see if some of them are vulnerable or not to
the X vulnerability. The only negative point is that these IDs are from
scripts input (user input).

For the solutions:
1) Creating our own reference is a good idea, but I think this will
   encourage people to just ignore this field.

2) The IDs are not limited to CVE or OSVDB, you can specify any value
   and in this case it would be better if the script writer just do it.
   There are a lot of IDs: Debian, Gentoo, FreeBSD ...

   Or we can just do: 'Program-2011-001', 'Program-2011-08-07' or
                      'Program-2011-08-07-1'

   Of course we'll document that this field should be updated to more
   popular IDs when they are available.


We'll see what others think of this.


> This probably goes outside the scope, but what would the XML output look
> like? It'd be great if we could somehow use the internal tags to create XML
> tags to easily identify the state/risk factor/references etc. (to save us
> from having to parse all of the script output first).
For the XML support currently it's not easy. You know that Nmap will
just put all the script output in the 'output' attribute of the 'script'
tag.

A clean solution would be to move all the NSE output code into a new file
nse_output.cc and then try to create and register XML output there.
Later the code will just inspect the registred XML data and write it
under the 'script' tag. If you want to regroup _all_ the vulnerabilities
XML output then a postrule script can do the job.

And it would be awesome if you can propose a first XML output sample, so
later we can start from it. Thanks in advance.

> I know Daniel Miller has suggested (and even supplied) a YAML based
> solution, but I find it easier to read a more traditional XML output, and I
> generally use XPath to extract data from XML files generated by other tools
> (at least Ruby has native YAML support, if I ever need to go that way). My
I just did a quick look at YAML specification and it seems that we can
support it. I remember that Daniel Miller submitted a patch but I don't
know it state, if I've time I'll try to look at it.

> concern is that the vulnerability data is crying out to be marked up to
> allow for easy data extraction, and without it we're not really improving
I agree.

> that much over the existing output, but if we hack in support for additional
> XML tags now for just the vulnerability data then would we make life more
> difficult if we later decided to introduce  YAML (or other) output for all
> NSE scripts. Personally, I like the idea of XML rather than YAML, especially
> as it allows us to easily validate the Nmap XML files. It also avoids mixing
> XML and YAML in the same file (I'd prefer it if the XML output just
> contained XML; if people want YAML due to its good data representation then
> maybe we should create a YAML output file?).
As I've said we should start with a clean approach, move the output code
into nse_output.cc file and do all the stuff there.

If we have a better XML support in NSE then I think that the YAML output
will not be mixed with the XML one.

> Rob 
Thanks for the feedback.

-- 
tixxdz
http://opendz.org
_______________________________________________
Sent through the nmap-dev mailing list
http://cgi.insecure.org/mailman/listinfo/nmap-dev
Archived at http://seclists.org/nmap-dev/