Really dumb questions?

[brian at fortifysoftware.com (Brian Chess)]

> - So when a vendor says that they are focused on quality and not
> security, and vice versa what exactly does this mean?

We spend most of Chapter 2 of Secure Programming with Static Analysis
describing the different problems that static analysis tools try to solve,
and we show where we think all of the companies you mention (plus a lot of
others) fit in.  The relative importance of false positives vs false
negatives is one important difference, but so is extensibility, rule set (as
John mentioned), ability of the tool to prioritize its findings, and the
interface the tool presents for exploring the results.  From my experience,
the vendors do different things in all of these areas, and these differences
aren't just a result of dumb luck.  They stem from different philosophies
about what the tools are supposed to do.  "Quality vs. Security" may be an
oversimplification, but the differences between the tools are much more than
cosmetic.


> - Is it reasonable to expect that all of the vendors in this space will
> have the ability to support COBOL, Ruby and Smalltalk sometime next year
> so that customers don't have to specifically request it?

I don't think so.  The way a tool is designed can make it easier or harder
to add support for a new language, but unless you're doing a really
superficial analysis, adding a new language is always a big deal. Supporting
a language requires more than just being able to parse it.  The tools often
have to do special work to make sure that the meaning of common idioms
carries over correctly in the analysis, then there's the small matter of
developing a rule set.

Someone mentioned that Ruby makes life hard because it lacks static types.
While that's true, it compensates in other ways.  For example, because of a
lack of static types, there are often more bugs to find.  There's some
really good academic work going on right now around security analysis of
scripting languages (mostly PHP).  Here's my pick of the week:

Sound and Precise Analysis of Web Applications for Injection Vulnerabilities
by Gary Wassermann and Zhendong Su
http://wwwcsif.cs.ucdavis.edu/~wassermg/research/pldi07.pdf


Regards,
Brian