Re: InformIT: comparing static analysis tools

[Chris Eng <ceng () veracode com>]

I'm not the Chris you posed the question to but I'll answer anyway.  :)

Usually the type of response you described is a knee-jerk reaction.  It's a different model than people are used to, 
and sometimes people are averse to change, whether that's warranted or not.  It's important to get past the initial 
reaction and actually have a substantive conversation.

Naturally, we try to understand each customer's specific hang-ups, but generally speaking there are a couple of things 
we always cover.  First, the customer needs to understand that they are NOT, in fact, uploading their code.  If they 
are used to using on-premise tools that require source code, they'll often make this mistake.  They are uploading 
binaries -- compiled code, or bytecode -- not their source.  Second, we have many layers of safeguards in place ranging 
covering process (Systrust), infrastructure (SAS-70 Type II), and of course application security itself (automated 
scanning plus manual penetration tests, multi-factor authentication, extremely granular roles and access controls, 
per-application backend encryption of results, flexible retention policies, etc.).  

Viewing this with a wider lens, there are a lot of factors involved in selecting a tool/service vendor.  One factor 
that comes into play for us is simply that our solution scales, and many others do not.  We can address the application 
supply chain problem in ways that others can't.  

-chris





Chris Eng
Senior Director, Research
Veracode, Inc.
Office: 781.418.3828
Mobile: 617.501.3280
ceng () veracode com 


-----Original Message-----
From: sc-l-bounces () securecoding org [mailto:sc-l-bounces () securecoding org] On Behalf Of Jim Manico
Sent: Thursday, February 03, 2011 7:02 PM
To: Chris Wysopal
Cc: Secure Code Mailing List
Subject: Re: [SC-L] InformIT: comparing static analysis tools

Chris,

I've tried to leverage Veracode in recent engagements. Here is how the conversation went:

Jim:
"Boss, can I upload all of your code to this cool SaaS service for analysis?"

Client:
"Uh no, and next time you ask, I'm having you committed".

I'm sure you have faced these objections before. How do you work around them?

-Jim Manico
http://manico.net

On Feb 3, 2011, at 1:54 PM, Chris Wysopal <cwysopal () veracode com> wrote:

> Nice article.  In the 5 years Veracode has been selling static analysis services we have seen the market mature.  In 
> the beginning, organizations were down in the weeds. "What false positive rate or false negative rate does the 
> tool/service have over a test suite such as SAMATE."  Then we saw a move up to looking at the trees.  "Did the 
> tool/service support the Java frameworks I am using?"  Now we are seeing organizations look at the forest. "Can I 
> scale static analysis effectively over all my development sites, my outsourcers, and vendors?"  This is a good sign 
> of a maturing market.
>
> It is my firm belief that software security has a consumption problem.  We know what the defects are.  We know how to 
> fix them.  We even have automation for detecting a lot of them.  The problem is getting the information and 
> technology to the right person at the right time effectively and managing an organization-wide program.  This is the 
> next challenge for static analysis. <bias-alert>I think SaaS based software is more easily consumed and this isn't 
> any different for software security</bias-alert>
>
> -Chris
>
> -----Original Message-----
> From: sc-l-bounces () securecoding org [mailto:sc-l-bounces () securecoding org] On Behalf Of Gary McGraw
> Sent: Wednesday, February 02, 2011 9:49 AM
> To: Secure Code Mailing List
> Subject: [SC-L] InformIT: comparing static analysis tools
>
> hi sc-l,
>
> John Steven and I recently collaborated on an article for informIT.  The article is called "Software [In]security: 
> Comparing Apples, Oranges, and Aardvarks (or, All Static Analysis Tools Are Not Created Equal)" and is available here:
> http://www.informit.com/articles/article.aspx?p=1680863
>
> Now that static analysis tools like Fortify and Ounce are hitting the mainstream there are many potential customers 
> who want to compare them and pick the best one.  We explain why that's more difficult than it sounds at first and 
> what to watch out for as you begin to compare tools.  We did this in order to get out in front of "test suites" that 
> purport to work for tool comparison.  If you wonder why such suites may not work as advertised, read the article.
>
> Your feedback is welcome.
>
> gem
>
> company www.cigital.com
> podcast www.cigital.com/silverbullet
> blog www.cigital.com/justiceleague
> book www.swsec.com
>
> _______________________________________________
> Secure Coding mailing list (SC-L) SC-L () securecoding org List information, subscriptions, etc - 
> http://krvw.com/mailman/listinfo/sc-l
> List charter available at - http://www.securecoding.org/list/charter.php
> SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the 
> software security community.
> Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates
> _______________________________________________
>
> _______________________________________________
> Secure Coding mailing list (SC-L) SC-L () securecoding org
> List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
> List charter available at - http://www.securecoding.org/list/charter.php
> SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
> as a free, non-commercial service to the software security community.
> Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates
> _______________________________________________

_______________________________________________
Secure Coding mailing list (SC-L) SC-L () securecoding org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates
_______________________________________________

_______________________________________________
Secure Coding mailing list (SC-L) SC-L () securecoding org
List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l
List charter available at - http://www.securecoding.org/list/charter.php
SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com)
as a free, non-commercial service to the software security community.
Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates
_______________________________________________