Re: CVSS is the worst compression algorithm ever

["Monroe, Bruce" <bruce.monroe () intel com>]

Hi Dave,

I participate on the CVSS SIG being ran out of FIRST that is working on improvements to CVSS. So do a number of people 
out of CERT CC, NIST, MITRE along with a good representation of industry. A number of us provided feedback on this 
paper. CVSS is for scoring the severity of a vulnerability. CVSS does not = Risk.

My understanding is there is a number of government entities that believe CVSS does = Risk and are using it in a vacuum 
for that purpose. While the CVSS score is a single component - you also must look at how the vulnerable component is 
deployed, controls in place, value of asset, patching windows, likelihood of exploit, ect...there is a lot that goes 
into determining risk.

The fact that various USG entities is using CVSS wrong is an education issue imo. Yes CVSS has it's issues with some of 
it's elements being subjective eye of the beholder type items but that isn't the reason for this paper...they've got 
USG people using it in a vacuum when it's only a single element of determining your orgs risk due to a vulnerability. 
That isn't a CVSS problem that's a vulnerability management 101 problem.

Regards,
Bruce
Intel PSIRT

Opinions expressed are my own and may not reflect those of my employer.
From: Dailydave <dailydave-bounces () lists immunityinc com> On Behalf Of Dave Aitel
Sent: Tuesday, January 08, 2019 8:14 AM
To: dailydave () lists immunityinc com
Subject: [Dailydave] CVSS is the worst compression algorithm ever


I wanted to take a few minutes and do a quick highlight of a paper from CMU-CERT which I think most people have missed 
out on: https://resources.sei.cmu.edu/asset_files/WhitePaper/2018_019_001_538372.pdf
Towards Improving CVSS - 
resources.sei.cmu.edu<https://resources.sei.cmu.edu/asset_files/WhitePaper/2018_019_001_538372.pdf>
resources.sei.cmu.edu
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY REV-03.18.2016.0 Distribution Statement A: Approved for 
Public Release; Distribution Is Unlimited TOWARDS IMPROVING CVSS

It's almost as funny a read as their previous best work on how "clientless HTTPS VPNs are insanely 
dumb<https://www.kb.cert.org/vuls/id/261869/> what were you thinking omg?"

They use a ton of big words in the paper to call CVSS out and give it a shellacking. Like most of you, we have 
extensive use of CVSS in our consulting practice and I've seen this stuff first hand. CVSS is of course just a buggy 
compression algorithm for taking complex qualitative data and then putting it on a number line. The paper has three 
angles here:

  1.  Qualitative mappings into quantitative numbers are a silly thing to do, like people trying to do "social science" 
by using SurveyMonkey.
  2.  We're pretty sure that the compression algorithm is not, in fact, putting higher risk items as bigger numbers, 
which is the whole point of the thing.
  3.  Nobody is applying this in any sort of consistent way (which is probably impossible) which is ALSO the whole 
point of the thing.

It's fine to have a lossy compression algorithm that emphasizes certain aspects of the input signal over others, of 
course, but an additional CERT/CC critique is we have no reason to think CVSS does this in any useful way.



There's definitely people in the CVSS process (who I will avoid calling out by name) who think ANY quantization is 
good. But read the paper and decide for yourself - because these are probably serious issues that are turning your 
entire risk org into a Garbage-In-Garbage-Out org...



-dave


_______________________________________________
Dailydave mailing list
Dailydave () lists immunityinc com
https://lists.immunityinc.com/mailman/listinfo/dailydave