Dailydave mailing list archives
Re: Coverage and a recent paper by L. Suto
From: "J.M. Seitz" <lists () bughunter ca>
Date: Mon, 29 Oct 2007 06:42:16 -0800
Honestly I don't think that the testing tools matter as much as the talent of their respective users. We've used a wide variety of tools and they're pretty much all "trying" to do the same thing. Automation == time savings && identification of low hanging fruit (not to mention false positives and false negatives). Automation != quality assessment && quality report, only talent can deliver that.
I agree wholeheartedly. However, I do think that you can get your automation inside a QA cycle to the point where you are deep-diving and finding the not-so-low-hanging fruit. This does require skill, budget and time to achieve. Internally, I have used some tools under trial as "first pass" scanners, to see what they found, and to be honest I wasn't overly impressed. In the automation cycles I have helped develop, you first test to the spec of the software (does it do what it's supposed to) and secondly you test the corner cases (robustness). From personal experience, this has worked wonders, and has measurably reduced the amount of bugs shipped. I also think that we are going to start seeing some integration consulting where software dev firms are going to begin hiring out for not only asessments on their products, but integrators and tool builders that can develop highly specific tools (fuzzers, scanners, etc.) that will integrate directly into their QA/automation cycles. For the shops I have done this for, I can tell you that there is no way that an out of the box solution would have found as many bugs as a custom built one. JS _______________________________________________ Dailydave mailing list Dailydave () lists immunitysec com http://lists.immunitysec.com/mailman/listinfo/dailydave
Current thread:
- Re: [fuzzing] Coverage and a recent paper by L. Suto, (continued)
- Re: [fuzzing] Coverage and a recent paper by L. Suto JFV (Oct 18)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Nicolas RUFF (Oct 25)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Alexander Sotirov (Oct 26)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Lance M. Havok (Oct 26)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Nicolas RUFF (Oct 27)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Lance M. Havok (Oct 27)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Alexander Sotirov (Oct 28)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Andre Gironda (Oct 29)
- Re: [fuzzing] Coverage and a recent paper by L. Suto J.M. Seitz (Oct 29)
- Re: Coverage and a recent paper by L. Suto J.M. Seitz (Oct 29)