Perspectives on Code Scanning

[James.McGovern at thehartford.com (McGovern, James F (HTSC, IT))]

It was bothering me for a long time that I didn't find evidence of any enterprise of size wanting to even purchase 
"seats" for source code scanning by developers yet I find tons of evidence that folks want to have deeper audits and 
have the tools in the hands of a few. One cannot ignore the importance of the thud factor when the report comes out 
especially in organizations that are led by process weenies who aren't really technical.
 
I do believe that enterprises would entertain tools that enabled secure design and figure out a way to apply security 
patterns such as what is outlined in the Core Security Patterns book. Tools that could somehow automate 
architecture/design reviews would be of immense value as well. 
 
One interesting thought I have is that more developers may care about secure coding if there were statistics that 
compared American developers to those employed in other countries. Right now, outsourcing is an unpopular topic where 
folks respond based on how they feel. Imagine the press if folks could factually prove that folks in other countries 
increase security defects...

-----Original Message-----
From: arian.evans at gmail.com [mailto:arian.evans at gmail.com]On Behalf Of Arian J. Evans
Sent: Thursday, June 07, 2007 4:21 PM
To: McGovern, James F (HTSC, IT); Secure Coding
Subject: Re: [SC-L] Perspectives on Code Scanning


<inline>


On 6/6/07, McGovern, James F (HTSC, IT) < James.McGovern at thehartford.com> wrote: 

I really hope that this email doesn't generate a ton of offline emails and hope that folks will talk publicly. It has 
been my latest thinking that the value of tools in this space are not really targeted at developers but should be 
targeted at executives who care about overall quality and security folks who care about risk. While developers are the 
ones to remediate, the accountability for secure coding resides elsewhere. 



Nice email James. These conversations are always enlightening. The responses tend to illuminate who has what experience 
types, between (a) university software experience, (b) government software-project experience, and (c) enterprise 
software experience. That makes a lot of difference in these discussions. Most enterprise /and/ small ISV developers I 
know, the good ones, either take pride in their quality of code and self-manage security issues, or are fast and 
productive and don't give a crap. 

And why should they give a crap? It's not their problem domain.

The armchair software-security pundits: "Shame on you. You didn't properly transcode these Hebrew and Latin code pages 
to avoid XSS attacks, dummy." 

The fast effective developer: "I delivered your functional specifications to the letter, on time, and the transcoding 
engine is FAST. What's the problem here?"




It would seem to be that tools that developers plug into their IDE should be free since the value proposition should 
reside elsewhere. Many of these tools provide "audit" functionality and allow enterprises to gain a view into their 
portfolio that they previously had zero clue about and this is where the value should reside. 



So of tools that plug into the IDE, let's distinguish between *source-code* and *run-time* "scanners". Source scanners 
I suspect will die a slow death, because sooner or later they are going to integrate into the IDE and per-seat value 
will plummet. They will be a "given feature" of IDEs. Sooner or later the IDE vendors will either buy & integrate, or 
come out with their own tools. Take Compuware, the quality is pretty low, but if I were a betting man I would bet that 
the bar gets set at "low-quality included-functionality" rather than set at "$50k per-seat amazing quality source code 
analzyer". 

I believe this is different from run-time or human design analysis, largely because of different business case.

For example: I have some clients that really like their Fortify tools, but they really don't like all the time and 
critical development resources it takes to use them, and how expensive they are. Cool tools, sexy technology, but they 
are hard to align with the business case and business goals for software development on multiple levels. 

Run-time analysis is different. Run-time "scanner" IDE-plugins as a concept is laughable, at best. Seriously -- who 
thinks that run-time scanners for developers is a viable idea?

 Run-time analysis's strengths are different too. It is easier at run-time to discover and analyze fundamental design 
flaws (note: I did not say "find them all", but definitely "find indication of them"), and to identify emergent 
behaviors. At best IDE plugins can perform some form of unit testing, but beyond verification of basic data-typing and 
IFOF/IFOE type issues, meh, not very useful. Not to mentioned entirely outside of the IDE problem-domain. 

Conclusion: two sets of problems, source analysis, and run-time analysis. I think there is a good-enough bar for source 
analysis that will get set fairly low and wind up baked into IDEs... similar to the Viz Studio /GS switch. I've already 
seen a pretty effective one that will probably wind up in one of the next releases of Visual Studio. It's actually 
better than some of the commercial offerings today, and baked in. 

Spot on James.

Human source analysis for Design Pattern issues, I think, this will always be needed. Same for run-time analysis. 
Different problem-domains they solve for though.




If there is even an iota of agreement, wouldn't it be in the best interest of folks here to get vendors to ignore 
developer specific licensing and instead focus on enterprise concerns?



So some marketing guy one day grokked "there are only n-number of security people per organization that scan run 
scanners, but there are Multiplier(n * 33.5) number of developers per organization....wow! We could sell all those 
developers scanners!!! Ka-Ching."
 
That sort of thinking is pretty cool, leads to some cool sales growth graphs and profitability projections. Need 
board-room meeting material, fits perfectly into that circular-arrow graph everyone has with "TCO" and "Lifecycle 
Management" and "ROI" and how it all loops together and saves everyone big bags of money after they spend up front. 

This circular "lifecycle-management" graph is labeled Security in the SDLC and shows how you can buy a lot of 
developer/IDE-scanners and have even the cheapest developers scan their code up front, and you'll save big in the long 
run by avoiding those expensive security audits, compliance issues, and having your smart (expensive) developers have 
to go back and fix all those holes. Ka-Ching. 

I haven't heard the business plans of any of the source-code-scanner vendors in a year or two...but back then, I didn't 
hear one that aligned with any enterprise software development or business plan reality that I have experienced. 

My thinking here could be off. I thought WAFs were a ridiculous idea but I'm starting to see very effective use-cases 
for them, despite the ridiculous claims and hyperbole that come from the WAF-vendor marketing nonsense. 

Anyway, I think you are spot on James, and I think if you look at Gartner's recent punditry you'll find some similar 
notions expressed.

Chow,


-- 
Arian Evans
software security stuff 






*************************************************************************
This communication, including attachments, is
for the exclusive use of addressee and may contain proprietary,
confidential and/or privileged information.  If you are not the intended
recipient, any use, copying, disclosure, dissemination or distribution is
strictly prohibited.  If you are not the intended recipient, please notify
the sender immediately by return e-mail, delete this communication and
destroy all copies.
*************************************************************************

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://krvw.com/pipermail/sc-l/attachments/20070607/d6777e5e/attachment-0001.html