Re: Rcov - interactive code coverage for fuzzers

[shadown <shadown () gmail com>]

Hi,

Actually there is a huge advantage when code coverage is done before fuzzing
because you spot the code that is unusually covered to manage your fuzzer to
cover those code sections, things like handshakings, commands, op codes of
the protocols, etc.
Another very interesting approach, when it comes to *static* analysis, is to
do the common searches for patterns of bugs that are not easily and then
combine the a code coverage tool like Paimei (which is an awesome tool)
looking for adjacent path to analysis what have to be inducted to reach the
section. This is something somehow obvious but there's a lot of researchers
that don't use this kind of approach.
And in all this cases code coverage is really useful.

Cheers,
 Sergio

PS: @Ari: yes, I what that private copy :)


On 3/27/07, Ari Takanen <ari.takanen () codenomicon com> wrote:
> Hello All,
>
> I think code coverage is a very interesting topic when combined with
> Fuzzing! A few comments below:
>
> On Mon, Mar 26, 2007 at 10:29:15PM -0400,
> dailydave-request () lists immunitysec com wrote:
> > From: Kowsik <kowsik () musecurity com>
> >
> > - Uses gcov's notes/data files to get at blocks and function summaries
>
> There is a very interesting Masters' Thesis on this topic made by one
> of our guys for Codenomicon (and University of Oulu) written in
> 2004. Basically as anyone with common sense can tell you, the research
> indicated that in some areas fuzzing has better code coverage than any
> other test technique, but also that code coverage gives no indication
> of the quality of fuzzing. I will see if we can publish that and I
> will send you the link on it later. Please let me know if you want a
> private copy.
>
> > From: Gadi Evron <ge () linuxbox org>
> >
> > We have three levels or layers (depends on approach):
> > 1. Building better fuzzers (which you cover).
> > 2. Helping the fuzzing process, fuzzing better.
> > 3. Making the process of finding the actual vulnerability once an
> >    indication is found (a successful test case, or as they say in QA, a
> >    passing one) easier.
>
> As I mentioned, code coverage indicates nothing about the quality of
> fuzzing. Well it can indicate that your fuzzing is very bad though if
> you do not get any coverage. But more importantly, I think code
> coverage is an excellent metric for "attack surface" i.e. when you
> fuzz you will discover the lines of code that are security critical. A
> study we did in 2004 definitely indicated that traditional QA did not
> touch large part of the code, whereas fuzzing (using Codenomicon
> tools) increased the amount of code touched during testing. I would be
> interested in seeing how this can be used to improve fuzzers, but more
> importantly this word should be sent to our friends in the white-box
> testing who have the challenges in auditing millions of lines of
> code. Any metric of what code can be exploited through different
> interfaces can prioritize the focus of those analysist tools, and
> therefore improve the code quality significantly. This is 'indication'
> of the real attack surface. Note that there are some very bad
> definitions of attack surface used by some researchers.
>
> > Working for a fuzzing vendor I am only too familiar with the Turing
> > halting problem and seeking reality in the midst of eternal runs,
> > but the most interesting thing I found in the past few months (which
> > wasn't technical) is the clash of cultures between QA engineers and
> > Security professionals. It will be very interesting to see where we
> > end up.
>
> Heh, nice to notice that there are three commercial fuzzer vendors
> involved in this discussion...
>
> There are two branches of negative testing:
> 1) Fuzzing starts from infinite and tries to learn intelligence
> 2) Robustness testing starts from finite and tries to add intelligence
>
> After some development both result in better tests in the end, and
> probably into one semi-random approach with a lot of intelligence on
> interface specifications and real-life vulnerability knowledge. Best
> approach is probably somewhere in the middle. For some reason security
> people start with the approach one (maybe because it is easiest for
> finding one problem), and it fits in that scene perfectly (given that
> you only need to find one flaw, and you can invoice for all the used
> time). Us QA people prefer the second approach as it fits perfectly to
> the existing QA processes (as the purpose is to find "all" flaws, and
> as you can basically invoice for the saved time). In my opinion, the
> Turing problem applies to approach one more than to the approach two.
>
> Sorry if I hurt any feelings again (I seem to do that every time I
> write to DailyDave)! As you can see, I tend to have very strong
> opinions against any randomness in fuzzing... But this is the result
> of 10+ years of research experience (6+ years commercially) in fuzzing
> with all possible variants... ;)
>
> Those of you that are planning to attend Blackhat Europe: Codenomicon
> will be there! Come and meet with us (maybe over a beer)!
>
> Best regards,
>
> /Ari
>
> --
> -o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-
> Ari Takanen                       Codenomicon Ltd.
> ari.takanen () codenomicon com       Tutkijantie 4E
> tel: +358-40 50 67678             FIN-90570 Oulu
> http://www.codenomicon.com        Finland
> PGP: http://www.codenomicon.com/codenomicon-key.asc
> -o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-o-
> _______________________________________________
> Dailydave mailing list
> Dailydave () lists immunitysec com
> http://lists.immunitysec.com/mailman/listinfo/dailydave



--
Sergio Alvarez
Security, Research & Development
IT Security Consultant
email: shadown () gmail com

This message is confidential. It may also contain information that is
privileged or otherwise legally exempt from disclosure. If you have received
it by mistake please let us know by e-mail immediately and delete it from
your system; should also not copy the message nor disclose its contents to
anyone. Many thanks.

_______________________________________________
Dailydave mailing list
Dailydave () lists immunitysec com
http://lists.immunitysec.com/mailman/listinfo/dailydave