Re: CVSS is the worst compression algorithm ever

[Adrian Sanabria <adrian.sanabria () gmail com>]

Our pentesters use DREAD, which I think most people have moved on from, but
at least the scoring is clear and consistent.

In addition to CVE being wrong on critical details, I've found that most of
ExploitDB isn't exploits. Many are vulnerability checks and almost all are
incorrectly entered. PrivEsc will be labeled RCE and RCE will be labeled
DoS. It's all a mess. If I had the resources to burn it all down and start
from scratch, I'd do it.

--Adrian

On Tue, Jan 8, 2019, 17:47 Nathaniel Ferguson <jferguson () 126 com wrote:

> > They use a ton of big words in the paper to call CVSS out and give it a
> shellacking. Like most of you, we have extensive use of CVSS in our
> consulting practice and I've seen this stuff first hand. CVSS is of course
> just a buggy compression algorithm for taking complex qualitative data and
> then putting it on a number line.
>
>
> Over the years I've worked at a few different consultancies and at least
> originally basically no one used any sort of standardized metric, the
> reports were generally humorous from a technical standpoint as the numbers
> were basically just made up and didn't adhere to even basic statistics
> methodologies-- we take the X and multiple it by Y and add the Z and
> there's your score! Some even plotted them along cartoon looking graphs and
> plots and my suspicion was that they were really included to give depth to
> the reports and break up the monotony of text on text on text. Oddly, I've
> never even worked at a place that described the methodology as outlined in
> their reports to their employees leaving some question as to how a
> methodology was to be implemented if only the client ever heard about it.
>
> In that sense, CVSS et al make some amount of sense, if nothing else it
> standardizes to a common metric and isn't what the sales guy or operations
> manager made up. Additionally, what a strange word-- shellacking, there is
> no 'k' in the word until its made into a present participle.
>
> > The paper has three angles here:
> > Qualitative mappings into quantitative numbers are a silly thing to do,
> like people trying to do "social science" by using SurveyMonkey.
>
> Which is what most people are or were selling.
>
> > It's fine to have a lossy compression algorithm that emphasizes certain
> aspects of the input signal over others, of course, but an additional
> CERT/CC critique is we have no reason to think CVSS does this in any useful
> way.
>
> Well there 's a missing line here, you can see it from the way that
> client-side attacks perverted the concept of remote and so they made them
> remote also instead of adding the new line to the plot. Because of stuff
> like this. everything is remote now which limits its usefulness. This
> doesn't even touch on the fact that most of the CVE database is basically
> wrong from submissions including very limited data, id est "memory
> corruption results in a DoS".
>
> Nathaniel
>
> 在 2019-01-09 00:14:00，"Dave Aitel" <dave.aitel () cyxtera com> 写道：
>
> I wanted to take a few minutes and do a quick highlight of a paper from
> CMU-CERT which I think most people have missed out on:
> https://resources.sei.cmu.edu/asset_files/WhitePaper/2018_019_001_538372.pdf
> Towards Improving CVSS - resources.sei.cmu.edu
> <https://resources.sei.cmu.edu/asset_files/WhitePaper/2018_019_001_538372.pdf>
> resources.sei.cmu.edu
> SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
> REV-03.18.2016.0 Distribution Statement A: Approved for Public Release;
> Distribution Is Unlimited TOWARDS IMPROVING CVSS
> It's almost as funny a read as their previous best work on how "clientless
> HTTPS VPNs are insanely dumb <https://www.kb.cert.org/vuls/id/261869/> what were
> you thinking omg?"
>
> They use a ton of big words in the paper to call CVSS out and give it a
> shellacking. Like most of you, we have extensive use of CVSS in our
> consulting practice and I've seen this stuff first hand. CVSS is of course
> just a buggy compression algorithm for taking complex qualitative data and
> then putting it on a number line. The paper has three angles here:
>
>    1. Qualitative mappings into quantitative numbers are a silly thing to
>    do, like people trying to do "social science" by using SurveyMonkey.
>    2. We're pretty sure that the compression algorithm is not, in fact,
>    putting higher risk items as bigger numbers, which is the whole point of
>    the thing.
>    3. Nobody is applying this in any sort of consistent way (which is
>    probably impossible) which is ALSO the whole point of the thing.
>
>
> It's fine to have a lossy compression algorithm that emphasizes certain
> aspects of the input signal over others, of course, but an additional
> CERT/CC critique is we have no reason to think CVSS does this in any useful
> way.
>
>
> There's definitely people in the CVSS process (who I will avoid calling
> out by name) who think ANY quantization is good. But read the paper and
> decide for yourself - because these are probably serious issues that are
> turning your entire risk org into a Garbage-In-Garbage-Out org...
>
>
> -dave
>
>
>
>
>
> _______________________________________________
> Dailydave mailing list
> Dailydave () lists immunityinc com
> https://lists.immunityinc.com/mailman/listinfo/dailydave
_______________________________________________
Dailydave mailing list
Dailydave () lists immunityinc com
https://lists.immunityinc.com/mailman/listinfo/dailydave