Penetration Testing mailing list archives
RE: Why eEye Retina (was MBSA scanner)
From: "Riley Hassell" <rhassell () eeye com>
Date: Thu, 22 Apr 2004 12:38:51 -0700
While it is often given as a reason that one tool is better than another, it simply doesn't follow that an aptitude for discovering new vulnerabilities in code is the same as an aptitude for discovering known vulnerabilities in running services in the real world. IMHO, the skills are related, but significantly different.
I strongly disagree. When we audit an application for flaws we have to learn everything about the applications. This includes learning the protocols it utilizes, the features and components it offers, and enables, as well as any other important characteristic of that product. Methods that do not require thorough knowledge of a software product are nearly over. As the security level of a product increases, so must the abilities and resources of the people that audit that software. Microsoft has come a long ways in the last several years. Finding flaws in their enterprise software isn't a 5 minute fault test, it typically requires a significant amount of time and effort. During this time we learn things about the behavior of the application that greatly aid us in quickly producing a remote check that does not require any authentication. For example: A vulnerability we discovered in IIS several years ago invoved the parsing of NULL bytes in header entity data. Microsoft eliminated this flaw by correcting the coding mistakes that were made. They also modified one of the routines responsible for the processing incoming HTTP requests so that it prohibited the use of NULL bytes in various portions of the client request. Due to this change we were able to determine the existence of a patch, or in this case a security roll-up. Over the last years we have time after time worked with vendors to eliminate security issues in their products, and with each product we discover and document these behavior differences so that our Retina team can quickly produce audits for our scanner.
In my mind the analogy is similar to that of the difference between medical research and surgery. People who practice one extremely well don't usually practice the other to the same level, even though the skills (though not necessarily the mindsets) required to perform both are somewhat similar in many cases. One just happens to be focused on discovering new techniques out in the world, and the other happens to be focused on saving lives.
In our case we *always* perform regression tests on security fixes released by vendors. During this time we often find more issues, thereby notifying the vendor once again. We also, as stated before document differences in image "library/executable" versions. We supply this information to our audit team, which then creates a remote check in our product. In a researcher's ideal world, they could sit around all day and just break other people's software, in reality any real company will see this as ludicrous. Application auditing, and even source auditing rarely are capable of generating enough income to justify the salaries of a decent research team.If you believe that vulnerability research is only done to generate press for a company then you need to spend a little more time in the industry. Very few people are going to buy a product soley because of their research department. On the flip side, very few companies will invest in a research department unless the research department offers a valuable service to the company. Breaking other people's software is not a valuable service. While it may be good for the namespace, it rarely produces any income. Researchers are typically better "attackers" than defenders, engineers are typically better "defenders" than the researchers. By having the best of both worlds you have the ability to strengthen the security of product more than the next company. In our case the research we do directly benefits the products, you know the ones that pay our rent. -R Riley Hassell Senior Research Associate eEye Digital Security rhassell () eeye com ________________________________ From: Mike Murray [mailto:mmurray () ncircle com] Sent: Wed 4/21/2004 8:33 PM To: Shawn Edwards; clarke-cummings () columbus rr com Cc: pen-test () securityfocus com Subject: RE: Why eEye Retina (was MBSA scanner) Let me state up front: I work for a competitor in the VA market, so I'm going to stay far away from any discussion on products, and try to stick with a bit of philosophy. I had one comment on something that Shawn said:
I know for a fact that they have some very skilled persons doing dev there. ... Just check some of their development discoveries that's gotta count for something!
While this is definitely an argument for the fact that a company has very smart people working for it (which is definitely not in question in eEye's case), I question the validity of the argument as far as the evaluation of a network VA tool. If the ability to discover new vulnerabilities were the gold standard for a good VA tool, we'd all be buying something that Dave Aitel wrote. While it is often given as a reason that one tool is better than another, it simply doesn't follow that an aptitude for discovering new vulnerabilities in code is the same as an aptitude for discovering known vulnerabilities in running services in the real world. IMHO, the skills are related, but significantly different. In my mind the analogy is similar to that of the difference between medical research and surgery. People who practice one extremely well don't usually practice the other to the same level, even though the skills (though not necessarily the mindsets) required to perform both are somewhat similar in many cases. One just happens to be focused on discovering new techniques out in the world, and the other happens to be focused on saving lives. My $0.02. M ------------------------------------------------- Michael Murray Director of Vulnerability and Exposure Research nCircle Network Security Office: 416-533-5305 ------------------------------------------------- ------------------------------------------------------------------------------ Ethical Hacking at the InfoSec Institute. Mention this ad and get $545 off any course! All of our class sizes are guaranteed to be 10 students or less to facilitate one-on-one interaction with one of our expert instructors. Attend a course taught by an expert instructor with years of in-the-field pen testing experience in our state of the art hacking lab. Master the skills of an Ethical Hacker to better assess the security of your organization. Visit us at: http://www.infosecinstitute.com/courses/ethical_hacking_training.html ------------------------------------------------------------------------------- ------------------------------------------------------------------------------ Ethical Hacking at the InfoSec Institute. Mention this ad and get $545 off any course! All of our class sizes are guaranteed to be 10 students or less to facilitate one-on-one interaction with one of our expert instructors. Attend a course taught by an expert instructor with years of in-the-field pen testing experience in our state of the art hacking lab. Master the skills of an Ethical Hacker to better assess the security of your organization. Visit us at: http://www.infosecinstitute.com/courses/ethical_hacking_training.html -------------------------------------------------------------------------------
Current thread:
- RE: Why eEye Retina (was MBSA scanner), (continued)
- RE: Why eEye Retina (was MBSA scanner) Cam Beasley, ISO (Apr 21)
- RE: Why eEye Retina (was MBSA scanner) Chris Hurley (Apr 21)
- RE: Why eEye Retina (was MBSA scanner) Lovrien, Scott (Apr 21)
- Re: Why eEye Retina (was MBSA scanner) Renaud Deraison (Apr 22)
- RE: Why eEye Retina (was MBSA scanner) Mike Murray (Apr 22)
- Re: Why eEye Retina (was MBSA scanner) Shawn Edwards (Apr 22)
- RE: Why eEye Retina (was MBSA scanner) Robert E. Lee (Apr 22)
- RE: Why eEye Retina (was MBSA scanner) Peter Benson (Apr 22)
- RE: Why eEye Retina (was MBSA scanner) Doty, Stephen (BearingPoint) (Apr 22)
- Re: Why eEye Retina (was MBSA scanner) Rainer Duffner (Apr 24)
- RE: Why eEye Retina (was MBSA scanner) Riley Hassell (Apr 22)
- Re: Why eEye Retina (was MBSA scanner) clarke-cummings () columbus rr com (Apr 23)
- RE: Why eEye Retina (was MBSA scanner) Steve Goldsby (ICS) (Apr 26)
- RE: Why eEye Retina (was MBSA scanner) Cam Beasley, ISO (Apr 21)