Building Security In vs Auditing

[gunnar at arctecgroup.net (Gunnar Peterson)]

> 1. I haven't gotten a sense that a bakeoff matters. For example, if I wanted
> to write a simple JSP application, it really doesn't matter if I use Tomcat,
> Jetty, Resin or BEA from a functionality perspective while they may each have
> stuff that others don't, at the end of the day they are all good enough. So is
> there really that much difference in comparing say Fortify to OunceLabs or
> whatever other tools in this space exist vs simply choosing which ever one
> wants to cut me the best deal (e.g. site license for $99 a year :-) ?

I recommend that companies do a bakeoff to determine

1. ease of integration with dev process - everyone's dev/build process is
slightly different

2. signal to noise ratio - is the tool finding high priority/high impact
bugs?

3.  remediation guidance - finding is great, fixing is better, how
actionable and relevant is the remediation guidance?

4. extensibility - say you have a particular interface, like mq series for
example, which has homegrown authN and authZ foo that you want to use the
static analysis to determine if it is used correctly. How easy is it
build/check/enfore these rules?

5. roles - how easy is it to separate out roles/reports/functionaility like
developer, ant jockey, and auditor?

6. software architecture span - your high risk/high priority apps are
probably multi-tier w/ lots of integration points, how much visibility to
how many integration points and tiers does the static analysis tool allow
you to see? How easy is it to correlate across tiers and interfaces?

-gp