Re: Coverage and a recent paper by L. Suto

["J.M. Seitz" <lists () bughunter ca>]

> Honestly I don't think that the testing tools matter as much 
> as the talent of their respective users. We've used a wide 
> variety of tools and they're pretty much all "trying" to do 
> the same thing. Automation == time savings && identification 
> of low hanging fruit (not to mention false positives and 
> false negatives). Automation != quality assessment && quality 
> report, only talent can deliver that.

I agree wholeheartedly. However, I do think that you can get your automation
inside a QA cycle to the point where you are deep-diving and finding the
not-so-low-hanging fruit. This does require skill, budget and time to
achieve. Internally, I have used some tools under trial as "first pass"
scanners, to see what they found, and to be honest I wasn't overly
impressed. In the automation cycles I have helped develop, you first test to
the spec of the software (does it do what it's supposed to) and secondly you
test the corner cases (robustness). From personal experience, this has
worked wonders, and has measurably reduced the amount of bugs shipped.

I also think that we are going to start seeing some integration consulting
where software dev firms are going to begin hiring out for not only
asessments on their products, but integrators and tool builders that can
develop highly specific tools (fuzzers, scanners, etc.) that will integrate
directly into their QA/automation cycles. For the shops I have done this
for, I can tell you that there is no way that an out of the box solution
would have found as many bugs as a custom built one.

JS

_______________________________________________
Dailydave mailing list
Dailydave () lists immunitysec com
http://lists.immunitysec.com/mailman/listinfo/dailydave