Re: Re: Fuzzing objdump (PR 17512) and readelf (PR 17531)

[Alexander Cherepanov <cherepan () mccme ru>]

On 2014-11-07 14:43, Yury Gribov wrote:
> On 11/07/2014 01:59 PM, Hanno Böck wrote:
> > Am Fri, 07 Nov 2014 13:08:09 +0300
> > schrieb Yury Gribov <y.gribov () samsung com>:
> >
> > > This looks rather impressive.  Have you considered automatically
> > > detecting duplicates by e.g. analyzing stacktraces?

I do it based on the output of valgrind.

> > american-fuzzy-lop kind of does that. It creates a hash among the code
> > path and groups fuzzing samples by that. That's quite convenient.
>
> [Cc-ing Binutils ML back again]
>
> Yeah, I think there was even an article in one of recent PLDIs which
> discussed different approaches to filtering duplicates arising in
> compiler fuzz testing (they did various combinations of stacktraces,
> Valgrind output, program coverage, etc.).  I was just curious how well
> this works for real world tasks like objdump crashes.

You can see for yourself. My recent (since 2014-11-03 21:17:35 UTC) 
attachments in the mentioned PRs are tarballs containing a file list.txt 
which lists various errors from valgrind (with distinct backtraces) and 
one sample hitting it for every error.

Later I started to include short statistics in a comment. My last 
attachment to PR 17512 lists the following errors for `objdump -x`:

Files: 11
Errors:
      1 Argument 'size' of function malloc has a fishy (possibly 
negative) value: ...
     63 Conditional jump or move depends on uninitialised value(s)
     16 Invalid read of size ...
      2 Invalid write of size ...
      1 Syscall param write(buf) contains uninitialised byte(s)
     17 Use of uninitialised value of size ...

oss-security, please note invalid writes and fishy arguments for malloc.

Back to real world deduping. IMHO it's not ideal but works quite well, 
e.g. you can get 10 files out of thousands. If you have hundreds of 
thousands or millions of crashers (which was trivial with objdump in the 
beginning) valgrind is too slow. Replacing it with gdb improve the 
situation (we loose full analysis but get the stacktrace for the crash 
faster).

--
Alexander Cherepanov