Comparing Scanning Tools

[gunnar at arctecgroup.net (Gunnar Peterson)]

Hi James,

I think you are right to look at it as economic issue, but the other factor
to add into your model is not just the short term impact to developer
productivity (which is non-trivial), but also the long term effects of
making decisions *not* to deal with finding bugs.

"Cleaning up data breach costs more than encryption

Protecting customer records is a much less expensive than paying for
cleanup after a data breach or massive records loss, research company
Gartner said. Gartner analyst Avivah Litan testified on identity theft
at a Senate hearing held after the Department of Veterans Affairs lost
26.5 million vet identities. "A company with at least 10,000 accounts to
protect can spend, in the first year, as little as $6 per customer
account for just data encryption, or as much as $16 per customer account
for data encryption, host-based intrusion prevention, and strong
security audits combined," Litan said. "Compare [that] with an
expenditure of at least $90 per customer account when data is
compromised or exposed during a breach," she added. Litan recommended
encryption as the first step enterprises and government agencies should
take to protect customer/citizen data. If that's not feasible,
organizations should deploy host-based intrusion prevention systems, she
said, and/or conduct security audits to validate that the company or
agency has satisfactory controls in place."
http://www.techweb.com/wire/security/188702019

Or, Brian Chess once pointed out:
" My favorite historical analogy this month is from medicine: it took
*decades* between the time that researchers knew that fewer people died if
surgeons washed their hands and the time that antisepsis was common in the
medical community.  That lag was entirely due to social factors: if it's
1840 you've been successfully practicing medicine for decades, why would you
want to change your routine?  And yet imagine a modern day surgeon who says
"I'm really busy today, so I'm going to save time by not scrubbing in before
I start the operation."  It's simply unthinkable.  Hopefully software
development is headed in the same direction, but on an accelerated
timetable."

-gp

On 6/7/06 4:08 PM, "McGovern, James F (HTSC, IT)"
<James.McGovern at thehartford.com> wrote:

> Thanks for the response. One of the things that I have been struggling to
> understand is not the importance of using such a tool as I believe they
> provide value but more of the fact that these tools may not be financial
> sustainable.
>
> Many large enterprises nowadays outsource development to third parties.
> Likewise, the mindset in terms of budgeting tends to eschew "per developer
> seat" tool purchases. Nowadays, it is rare to find an enterprise not using
> free tools such as Eclipse and not paying for IDEs
>
> I have yet to find a large enterprise that has made a significant investment
> in such tools. I wonder if budgets and the tools themselves are really causing
> more harm than helping in that enterprises will now think about trading off
> such tools vs the expense they cost.
>
> -----Original Message-----
> From: leichter_jerrold at emc.com [mailto:leichter_jerrold at emc.com]
> Sent: Wednesday, June 07, 2006 4:34 PM
> To: McGovern, James F (HTSC, IT)
> Cc: sc-l at securecoding.org
> Subject: Re: [SC-L] Comparing Scanning Tools
>
>
> | Date: Mon, 5 Jun 2006 16:50:17 -0400
> | From: "McGovern, James F (HTSC, IT)" <James.McGovern at thehartford.com>
> | To: sc-l at securecoding.org
> | Subject: [SC-L] Comparing Scanning Tools
> | 
> | The industry analyst take on tools tends to be slightly different than
> | software practitioners at times. Curious if anyone has looked at Fortify
> and
> | has formed any positive / negative / neutral opinions on this tool and
> | others...
> We evaluated a couple of static code scanning tools internally.  The
> following is an extract from an analysis I did.  I've deliberately
> omitted comparisons - you want to know about Fortify, not how it
> compares to other products (which raises a whole bunch of other
> issues), and included the text below.  Standard disclaimers:  This
> is not EMC's position, it's my personal take.
>
> Caveats:  This analysis is based on a 3-hour vendor presentation.  The
> presenter may have made mistakes, and I certainly don't claim that my
> recall of what he said is error-free.  A later discussion with others
> familiar with Fortify indicated that the experience we had is typical,
> but is not necessarily the right way to evaluate the tool.  Effective
> use of Fortify requires building a set of rules appropriate to a
> particular environment, method of working, constraints, etc., etc.
> This takes significant time (6 months to a year) and effort, but
> it was claimed that once you've put in the effort, Fortify is a
> very good security scanner.  I am not in a position to evaluate that
> claim myself.
>
> BTW, one thing not called out below is that Fortify can be quite slow.
> Our experience in testing was that a Fortify scan took about twice as
> long as a C++ compile/link cycle, unless you add "data flow" analysis -
> in which case the time is much, much larger.
>
> The brief summary:  In my personal view, Fortify is a worthwhile tool,
> but it would not be my first choice.  (Given the opportunity to choose
> two tools, it would probably be my second.)  Others involved in the
> evaluation reached the opposite conclusion, and rated Fortify first.
>
> -- Jerry
>
> Fortify
>
>
> *************************************************************************
> This communication, including attachments, is
> for the exclusive use of addressee and may contain proprietary,
> confidential and/or privileged information.  If you are not the intended
> recipient, any use, copying, disclosure, dissemination or distribution is
> strictly prohibited.  If you are not the intended recipient, please notify
> the sender immediately by return e-mail, delete this communication and
> destroy all copies.
> *************************************************************************
>
>
> _______________________________________________
> Secure Coding mailing list (SC-L)
> SC-L at securecoding.org
> List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
> List charter available at - http://www.securecoding.org/list/charter.php