Full Disclosure mailing list archives
Was: Full Disclosure = Exploit Release - No disclosure No Fix
From: "yossarian" <yossarian () planet nl>
Date: Thu, 30 Jan 2003 23:05:55 +0100
yossarian wrote:What info do we need? I think their must be a reasonable amount of intelligence here, maybe we can - like the IETF proposal I mentioned earlier - combine some efforts, and get it over with. If we can settle
on
the major questions, like the one Paul Schmehl posted, we can try to get some real answers. Making a list of several hundred unsolved issues where notification took place, or fixes took ages, can be done in a few weeks. In risk
migitation, I
prefer to work with profiles - not only of attackers, but also of
vendors.
It should be possible to do in reality what Gartner did for show, in
2000,
when they said that by the end of 2002 MS would have become better at writing secure IIS, probability 0.8. Sure it is a lot of work, and
certainly
classification of unfixed, late fix, feasibility of fix in set time etc
will
be a nightmare, and the risk of lawsuits is real, but the benefits would
be
great. For the bigger vendors, statistics will iron out mistakes - and
its
the trends that count. Maybe they are getting better at it, who knows?
And a
propos the lawsuits, we don't all live in the USA.
BB wrote
Something like that has been at the top of my list for some time, when I get some free time and corresponding motivation. :) Securityocus used to have a decent set of simple statistics for how many vulns there were per vendor per time period. I wanted to add on to that the time to patch.
So,
take all the vulns for a particular set of vendors for say 2002. Look up when they were publicly announced. Perhaps see if you can determine when they were discovered and reported, too. Then see how long it took for the patch to be made available. Microsoft wouldn't look too terrible under those circumstances, except for perhaps the unpatched IE bugs. Sun would probably look horrible.
Maybe standardisation in disclosure would help the statistically inclined: OK, so what would be useful: date of discovery, date of 1st notification, date and summary of response, date of fix release - if any, single point of security notification at Vendor Y/N. This should be the bare minimum, I guess. Quality of fix would be nice, since some fixes don't fix. Fix vs. workaround could be useful, disabling a certain feature is IMHO no fix, even if the feature is generally considered useless - it might be useful to some, if it ain't, remove it. I don't think a server containing this info should be in the US, since it will be damaging to certain commercial interests.
That probably would help a lot in terms of pciking a vendor. I don't know that it addresses the question of whether exploits should be released. Perhaps it says that it's unfair to release Solaris exploits? :)
Statistics usually shed some light on the whereabouts of the facts. It will not give a direct answer as to the effectiveness of releasing sploits directly, but what it can do, is making it more effective, since it will single out bad vendors, and give them an opportunity to better themselves. If is a benchmark, vendors want the highest grades - it should competative. Make it grades, so the impact will be clear, even to sales and management ;--)
Back to the original point... what kinds of things do people want to know in order to take one side or another on whether full-disclosure is a good
idea:
-What do vendors do when they aren't forced to patch things by public disclosure? We know what they did many years ago, but now that they "know better" what would they do if full-disclosure went away?
Hard to answer, these what if questions.
-How often are vulnerabilities independently discovered/exploited/shared
by
bad guys? Define bad guys. We only know of a few instances here and there, and have the bragging of some vocal blackhats that can't be considered particularly trustworthy.
Well, you'd have to monitor all covert channels on the web to find out. Maybe Carnivore will help? Nah. Not realistic, i suppose.
-How often did someone "need" someone else's script to break into a box?
With only a estimated meagre 2% of attacks detected, who can tell? Define attack. And not all attacks are scriptable, only certain types - you can only script for flaws in systems used, not for all possible network design or system implementation errors. Could you imagine an midsize IT company setting up VPN to its corporate network without enabling encryption - I have seen it. Can you imagine a major Telco using open shares where any connected user has admin - No you wouldn't but it has happened - and has not changed yet. Would you write a script for that - unlikely. Competent people usually cannot foresee nor understand the errors made by the others. Of course some people need scripts, but stuff like nessus is hardly usefull unless you can code your own script. At this moment, Nessus can test 1165 different vulns, gathered over the years. I once did some research, and I found out that over a period of 6 months, 588 new vulnerabilities were posted on major lists. (april - sept 2001). So Nessus holds less than 1 years worth of vulns announced. Of course these are testing scripts, not actual sploits, but count the number of sploits on supposedly black hat servers. At most 200 per year emerge. Many go for the same vulns, quite a few do not work under any circumstances. Maybe the working ones are used to break hundreds of boxes, who knows - but not likely. I think it is very hard to prove the relation between available scripts - mostly just probing scripts - and actual attacks. Unless you consider a probe an attack. If it is very hard to prove, chances are (Occams Razor) that there is no relation. The only case you can make, is for viruses, where toolboxes can empower the dim. But this is not breaking into a box.
-How would the patch rate of customers change with more or less disclosure/hype?
Would be Nice to know, but I think to hype does not help. Where do these hypes happen - on dedicated lists. They are not read by the average admin. To this list of unanswereable questions I could add the ratio of security fixes with or without preceding full or half-full disclosure. BTW, whatever happened to disclosing 'somewhere inbetween', where only skilled people could understand the technical details of an advisory and turn it into a script? So I guess we by this venue can never prove if full disclosure is a good idea. But maybe it is not the correct question - we want vendors to build safer software, not prove we can find holes and quibble about the credits, commercial interests etc. Full disclosure is just a means to an end, and we cannot see the end getting any closer. Hence my suggestion of a benchmark, available to all. Some questions/issue's on my list for the benchmark. How do we make it a benchmark (i.e. understandable for the many, and useable for the consultant types). A software company, will it be judged on the number of programs they sell, the number of lines of code (say security hole per 1000 lines of code?), should the type of program be taken into account - with a reverse bonus if a program does not need external communication but still has remotely exploitable holes? (completely flawed versus just security flawed). Other probs Researchers follow trends and find the type of vulns they are looking for. Remember when embedded or default password were 'in'. Hardly see them anymore - now it has to be buffer overflows, header manipulation or unicode related. Does this mean the older types of flaws aren't there anymore? If true, it would prove that the industry is getting better at securing their products. If false, it just proves the few researchers find flaw any direction they care to look at, and many types of vulns will pass completely unnoticed, since they can only look in one direction at a time. Another major security problem I have never seen adressed here, nor anywhere else, is that the majority of people on the web is no longer native english speaking. (US 40%, plus some in UK, Canada, Down Under and in India, might just be half. Is this a security problem? I have noticed from reading security lists in other languages, that sometimes the vulns are not at all posted on the big, english forums. This will increase in the future. How to explain a complex hole in some american software to a vendor, if you can barely manage the language? They will not bother. So in the future, the vendor notification will drop. Research of few years ago (1998 if I am correct) proved that systems are significantly less patched in countries were use of english is less common. Server software, databases - many of the common programs or books are not translated. Advisories are not translated. Bugtraq, this list and readme files with updates and hotfixes are in english. I have done networks in Italy and France - well, the software was in English but the admins could rarely make a remotely understandable sentence, let alone notify a vender (or read the manuals). When the software was in the native language, there were no language specific patches, and the english patches didn't work properly, or changed part of the language in the menu's to english - or both. The security industry is also mainly american, or english-centered. The SQL worm proved again that secured shops can be hurt by unpatched systems elsewhere. Security has always been an uphill battle, but it is getting steeper. The security of the Net therefore will decrease, if vendors don't extend the support to other languages, and independent books are not translated - since it is not commercially viable. And the feasability of a complete list of known exploits will also decline. If 1 million chinese know, but no one cared to translate, does it qualify as a know exploit, or will it be a 0Day? My guess is that the best thing security researchers can do for the long term, is learn chinese. Especially if the corporate marketing strategy prescribes postings vulns - translating is a lot easier than crunching code. _______________________________________________ Full-Disclosure - We believe in it. Charter: http://lists.netsys.com/full-disclosure-charter.html
Current thread:
- Fw: Full Disclosure != Exploit Release - No disclosure No Fix yossarian (Jan 29)
- Re: Fw: Full Disclosure != Exploit Release - No disclosure No Fix Blue Boar (Jan 29)
- Message not available
- Message not available
- Was: Full Disclosure = Exploit Release - No disclosure No Fix yossarian (Jan 30)
- Re: Was: Full Disclosure = Exploit Release - No disclosure No Fix Blue Boar (Jan 30)
- Message not available
- Re: Fw: Full Disclosure != Exploit Release - No disclosure No Fix Blue Boar (Jan 29)