Re: From blackbox to grey-box during Web App tests

[Matt Hargett <matt () use net>]

On Friday 12 October 07 07:54:34 J.M. Seitz wrote:
> > PaiMei and BinNavi are fuzzer trackers, as I explained.  gcov
> > is more of a basic line coverage tool, isn't it?  See:
> > http://bullseye.com/coverage.html

The problem I always have with gcov is that it's output is too low-level and 
that aren't any good visualization tools for the data. ggcov doesn't count 
because it basically won't compile on any Linux Luis and I tried unless you 
sacrifice both a chicken *and* a goat. Even when we got ggcov compiled, it 
didn't work properly. (WAS RE: MY KINGDOM FOR A HORSE)

valgrind has all the capabilities to put this kind of information together, 
someone just needs to do it. I'll pay $250 for someone to develop an 
open-source GPLv2 line-based coverage "skin" for valgrind that outputs in 
NCover's XML format.

The other problem with gcov is that you have to recompile the source 
with -fprofile-arcs, etc. The fact it doesn't measure unique path coverage 
isn't a big surprise, because most of these tools don't do that. When working 
on Hailstorm 3.0 (in late 2001, I think? My co-workers at that time might 
remember.), I ran gcov on TinyHttpd to make sure the HTTP content in that 
product was covering the protocol parsing code correctly. The content was 
originally developed against the RFC, but the coverage wasn't very complete. 
We developed more content as a result. The key here was I was only caring 
about the coverage on the protocol parsing code itself, and I had to look at 
it deeply to weed out things that didn't matter. Code coverage as a raw 
metric is basically useless, but the understanding what *isn't* covered is a 
good place to start looking for bugs. The best exploitable application-logic 
bugs I've found in the last 5 years have had to do with poor exception 
handling in C++, Java, C#, and even a couple of custom-longjmp exception 
implementations and signal handling in plain-old C.


> Gcov is a whitebox code coverage tool, and there are some interesting tools
> from Compuware that integrate directly into your VS 2005 environment to
> give you code coverage metrics.

I've always preferred Parasoft's tools (Insure++, etc) for these kinds of 
things as their support has always been quite responsive with fixes for 
issues, regardless of the size of the company I was at. I recommend Gimpel's 
PC-Lint for the same reason. I'm not affiliated with either of the those 
companies, I've just been using their products off and on for almost 10 
years. (I did used to use PureCoverage, but like it's Purify cousin, the 
product has largely atrophied.)

> I guess I am not too sure how useful that Tracer is, the more I have
> thought about it, the more I like Dave's idea of having a filter on the
> running server (database), that captures all SQL queries that made it into
> the server, and reporting the results back that match an injection. To take
> this a step further you could also hook into the application server itself
> and hook file creation, process creation, etc.

I would recommend just doing destructive injection. If people don't have a 
staging system you can skullfuck with all your might, your ability to make 
the app fall over in a way that will get them to actually fix it is severely 
impaired.

Specifically with regard to tracer, I wasn't very impressed when an 
acquaintance of mine who works for Fortify showed it to me. He was equally 
underwhelmed, but the strategy of diversifying their business is a good one. 
One wonders how things would have played out different for Sanctum (or 
Cenzic) if they had diversified their product line sooner.
_______________________________________________
Dailydave mailing list
Dailydave () lists immunitysec com
http://lists.immunitysec.com/mailman/listinfo/dailydave