Re: "Back with another one of those block rockin' beats"

[Michael Eddington <meddington () gmail com>]

I'm curious, how long does your coverage trace of winword.exe with a 20
page or so word document take?  My own PIN based coverage tools take a
long time (minutes), which makes doing coverage of 10,000 files take a
while :)

mike

On 10/28/2010 9:02 AM, Ben Nagy wrote:
> Oh Dear.
>
> I certainly don't make it my business to be mean to people on mailing
> lists, but I am not hewn from stone. Perhaps it was the
> self-congratulatory tone that irked me, perhaps it was merely a bad
> phase of the moon...
>
> On Thu, Oct 28, 2010 at 12:42 AM, Henri Lindberg
> <henri+dailydave () nsense fi> wrote:
> > Good news everyone!
> > nSense is releasing a tool which instruments executables during
> > runtime in order to extract code coverage data.
> As so often happens when you show your tool to a large group of
> people, there are liable to be ... mixed reactions.
>
> > Why?
> > Analyzing code coverage of large or self modifying executables with
> > various input files is a non-trivial task.
> As is now evident.
>
> > The tool is written in Python and is released under the Beerware license[1].
> > [1] http://en.wikipedia.org/wiki/Beerware
> Regrettably, based on this, you will have to rely solely on your charm
> and good looks for those free beers.
>
> I wouldn't have replied, had we not recently scoped and written a
> 'code coverage tool designed for large executables' as part of my
> effort to bring Charlie Miller's leet fuzzing template selection
> techniques to closed source Windows targets. I'll be talking about all
> that more at the (awesome) Ruxcon in Australia in a few weeks. You
> should all go - other than me, the talk schedule is chock full of
> awesome. When I say 'we wrote' I really mean 'grugq wrote' by the way.
>
> We ran this against WINWORD.EXE... actually to be honest, we fixed it
> first, it doesn't seem to work out of the box... anyway, we ran it
> against Word and got 8 blocks. It was very fast though! Based on our
> ccov tracer, we think it should be more like 300k, depending on the
> input file, but of course we're not here to quibble over numbers.
>
> In an uncharacteristic burst of pity, The Grugq tested it against the
> old standby, calc.exe, a much softer target. (As a tip for young
> players, please test your binary analysis tools against things OTHER
> than calc.exe.)
>
> blocks.py
> "instruction mode": real: 9.59 seconds, 269 basic blocks
> "dump mode": real: 1m53s, 543 basic blocks
>
> Our (PIN based) ccovtrace.dll
> time: 9.67 seconds  34385 basic blocks
> Of which, 1222 inside calc.exe
>
> So why the differences?
>
> Based on an initial look, it seems there are major flaws with the
> instrumentation algorithm:
>
> def read_jump(self):
>    # ref
>    if self.op1[0] == '[' and self.op1[1] != 'e':
>        (...instrument stuff)
>    # value
>    if self.op1[0] != '[' and len(self.op1) > 3  and self.op1[0] != 'e':
>        (...instrument stuff)
>
> If I'm not missing something, this doesn't pick up call [eax] etc, which
> is not exactly uncommon. Also, things like jnz <byte> won't be
> instrumented, and so on.
>
> This is also baffling:
>
> def is_inscope(self, entry_point):
>    if self.base <= entry_point < self.base + self.size:
>            return True
>
> It is actually sometimes useful to trace DLLs, you know, and if you
> are planning to trace 'malicious code' then it could be running more
> or less anywhere.
>
> So, due to the incremental (and bad) nature of the instrumentation,
> the instant you miss ONE control transfer, which you will, you lose all
> control of the executable. As a side note, if you are doing binary
> analysis with strcmp in 2010 then "you're doing it wrong".
>
> Anyway, please, of course, accept my abject apologies if we somehow
> introduced crippling bugs as a result of fixing your indenting (I've
> heard python is like that) and all of these results are wrong.
>
> Enjoy T2.
>
> ben
>
> PS: YES, we will release our tracing / ccov tools, before you ask. You
> can have your turn to laugh after rux.
> _______________________________________________
> Dailydave mailing list
> Dailydave () lists immunitysec com
> http://lists.immunitysec.com/mailman/listinfo/dailydave
_______________________________________________
Dailydave mailing list
Dailydave () lists immunitysec com
http://lists.immunitysec.com/mailman/listinfo/dailydave