Dailydave mailing list archives
Re: [fuzzing] Coverage and a recent paper by L. Suto
From: Alexander Sotirov <alex () sotirov net>
Date: Sun, 28 Oct 2007 14:28:15 -0700
On Sat, Oct 27, 2007 at 09:25:47AM +0200, Nicolas RUFF wrote:
Using the following perl script two buffer overflows are detected: cat vuln.c | perl -ne '/rnd\[i\]/ and print "Buffer overflow!\n"' This post does have a point. Discuss among yourselves.Is this vendor bashing, maybe ? ;)
Not at all. I've written static analysis tools myself and I know how hard a problem it is, so I have nothing but respect for the people trying to solve it. My point is that comparing static analysis tools by testing them on a single vulnerable function is a very poor way to test their performance. It is very easy to construct samples that will show the strenghts of one tool and the weaknesses of the others (see my vulncheck paper for a great example of that), as well as to write a Perl one-liner that will beat all commercial tools when run on a single program. It's a very similar situation to compiler benchmarking. Microbenchmarks are a great way to test specific types of optimizations, but they don't reflect the real-world preformance of the compiler. The only way to compare static analysis tools is to use a large sample set of real vulnerabilties and measure the false positive and false negative rates. Everything else is a waste of time. Alex
Attachment:
_bin
Description:
_______________________________________________ Dailydave mailing list Dailydave () lists immunitysec com http://lists.immunitysec.com/mailman/listinfo/dailydave
Current thread:
- Re: Coverage and a recent paper by L. Suto, (continued)
- Re: Coverage and a recent paper by L. Suto matthew wollenweber (Oct 15)
- Re: Coverage and a recent paper by L. Suto Stephen John Smoogen (Oct 16)
- Re: Coverage and a recent paper by L. Suto Charles Miller (Oct 16)
- Message not available
- Message not available
- Re: [fuzzing] Coverage and a recent paper by L. Suto matthew wollenweber (Oct 17)
- Re: [fuzzing] Coverage and a recent paper by L. Suto JFV (Oct 18)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Nicolas RUFF (Oct 25)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Alexander Sotirov (Oct 26)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Lance M. Havok (Oct 26)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Nicolas RUFF (Oct 27)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Lance M. Havok (Oct 27)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Alexander Sotirov (Oct 28)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Andre Gironda (Oct 29)
- Re: [fuzzing] Coverage and a recent paper by L. Suto J.M. Seitz (Oct 29)
- Re: Coverage and a recent paper by L. Suto matthew wollenweber (Oct 15)
- Re: Coverage and a recent paper by L. Suto J.M. Seitz (Oct 29)