Full Disclosure mailing list archives
RE: windows linux final study
From: security curmudgeon <jericho () attrition org>
Date: Tue, 29 Mar 2005 17:06:26 -0500 (EST)
: Yes, but did you actually verify their research using their methodology : to see if they screwed up? Personally no, and I won't =) : As in any study, the methodology and assumptions control the result. You : can either poke holes in their methodology (for example by pointing out : that the use of only published results is not a true indication of their : security, in which case the eEye list of purported flaws is relevant) or : you can use their exact methodology to recreate the work and prove that : their data collection was wrong. Exactly. Perhaps I should have expanded on that toward the end, but my comment about mincing words refers to this. Like the previous Microsoft funded report in 1999 [1], a company named Mindcraft set up a test to compare Linux vs Windows NT Server 4.0 and judge their "performance" (which was also rebutted fairly well [2]). The two things that Mindcraft didn't mention at first were 1) Microsoft funded the test and 2) Mindcraft's advertising completely destroyed their own credibility: "With our custom performance testing service, we work with you to define test goals. Then we put together the necessary tools and do the testing. We report the results back to you in a form that satisfies the test goals." So like you said, if you design the methodology with the intent of reaching certain results, you can't trust it as anything but a glorified marketing brochure. Instead of pointing that out more clearly (as I probably should have), I skipped past that to point out Microsoft is not fast to patch. Of course they have a < 30 day response time when they coordinate disclosure with these companies, many of which get advanced copies of their software and don't want to burn bridges, and many of which follow ethical disclosure (to a fault sometimes). In the real world, discovery of a vulnerability isn't limited to one person or company. We have seen this occur several times, where NGSS, eEye and iDefense [3] discovered the same vulnerability, and the resulting advisory w/ patches came out months later [4]. During that time frame, we can't expect or assume that no one else discovered the issue and/or used it for their own ends. Focusing on the time between public disclosure and patch release is a red herring (in this case). : I guess what I'm saying is that you can't say the study is wrong if they : release and follow their own methodology, but you CAN say its just plain : not relevant due to the assumptions and methodology. And, lets not : forget, the person who FUNDS the study probably CONTROLS the : assumptions! In an ideal academic setting, I can't say the study is wrong if their methodology is followed and results published. In reality, if I factor in other things (like you just did with RedHat and 'official' patches), I can say the study is wrong. There is often a serious disconnect between academia and the rest of the world who actually deploys these platforms, and that is lost in these 'studies'. jericho [1] http://www.mindcraft.com/whitepapers/nts4rhlinux.html [2] http://lwn.net/1999/features/MindCraft1.0.php3 [3] http://www.nextgenss.com/advisories/realra.txt http://www.idefense.com/application/poi/display?id=109&type=vulnerabilities http://www.eeye.com/html/research/advisories/AD20040610.html [4] http://www.service.real.com/help/faq/security/040610_player/EN/ _______________________________________________ Full-Disclosure - We believe in it. Charter: http://lists.grok.org.uk/full-disclosure-charter.html Hosted and sponsored by Secunia - http://secunia.com/
Current thread:
- Re: windows linux final study, (continued)
- Re: windows linux final study Vladamir (Mar 27)
- RE: windows linux final study ChrisDay (Mar 29)
- RE: windows linux final study David Solbach (Mar 29)
- RE: windows linux final study David Solbach (Mar 29)
- RE: windows linux final study security curmudgeon (Mar 29)
- Re: windows linux final study Florian Weimer (Mar 29)
- Re: windows linux final study Joerg Kurz (Mar 29)
- Re: windows linux final study Ron (Mar 29)
- RE: windows linux final study David Solbach (Mar 29)
- RE: windows linux final study ChrisDay (Mar 29)
- RE: windows linux final study Lachniet, Mark (Mar 29)
- RE: windows linux final study security curmudgeon (Mar 29)
- RE: windows linux final study Ron DuFresne (Mar 29)
- RE: windows linux final study Mark J Cox (Mar 30)