Perspectives on Code Scanning

[arian.evans at anachronic.com (Arian J. Evans)]

<inline>

On 6/6/07, McGovern, James F (HTSC, IT) <James.McGovern at thehartford.com>
wrote:
> I really hope that this email doesn't generate a ton of offline emails and
> hope that folks will talk publicly. It has been my latest thinking that the
> value of tools in this space are not really targeted at developers but
> should be targeted at executives who care about overall quality and security
> folks who care about risk. While developers are the ones to remediate, the
> accountability for secure coding resides elsewhere.



Nice email James. These conversations are always enlightening. The responses
tend to illuminate who has what experience types, between (a) university
software experience, (b) government software-project experience, and (c)
enterprise software experience. That makes a lot of difference in these
discussions. Most enterprise /and/ small ISV developers I know, the good
ones, either take pride in their quality of code and self-manage security
issues, or are fast and productive and don't give a crap.

And why should they give a crap? It's not their problem domain.

The armchair software-security pundits: "Shame on you. You didn't properly
transcode these Hebrew and Latin code pages to avoid XSS attacks, dummy."

The fast effective developer: "I delivered your functional specifications to
the letter, on time, and the transcoding engine is FAST. What's the problem
here?"


It would seem to be that tools that developers plug into their IDE should be
> free since the value proposition should reside elsewhere. Many of these
> tools provide "audit" functionality and allow enterprises to gain a view
> into their portfolio that they previously had zero clue about and this is
> where the value should reside.



So of tools that plug into the IDE, let's distinguish between *source-code*
and *run-time* "scanners". Source scanners I suspect will die a slow death,
because sooner or later they are going to integrate into the IDE and
per-seat value will plummet. They will be a "given feature" of IDEs. Sooner
or later the IDE vendors will either buy & integrate, or come out with their
own tools. Take Compuware, the quality is pretty low, but if I were a
betting man I would bet that the bar gets set at "low-quality
included-functionality" rather than set at "$50k per-seat amazing quality
source code analzyer".

I believe this is different from run-time or human design analysis, largely
because of different business case.

For example: I have some clients that really like their Fortify tools, but
they really don't like all the time and critical development resources it
takes to use them, and how expensive they are. Cool tools, sexy technology,
but they are hard to align with the business case and business goals for
software development on multiple levels.

Run-time analysis is different. Run-time "scanner" IDE-plugins as a concept
is laughable, at best. Seriously -- who thinks that run-time scanners for
developers is a viable idea?

 Run-time analysis's strengths are different too. It is easier at run-time
to discover and analyze fundamental design flaws (note: I did not say "find
them all", but definitely "find indication of them"), and to identify
emergent behaviors. At best IDE plugins can perform some form of unit
testing, but beyond verification of basic data-typing and IFOF/IFOE type
issues, meh, not very useful. Not to mentioned entirely outside of the IDE
problem-domain.

Conclusion: two sets of problems, source analysis, and run-time analysis. I
think there is a good-enough bar for source analysis that will get set
fairly low and wind up baked into IDEs... similar to the Viz Studio /GS
switch. I've already seen a pretty effective one that will probably wind up
in one of the next releases of Visual Studio. It's actually better than some
of the commercial offerings today, and baked in.

Spot on James.

Human source analysis for Design Pattern issues, I think, this will always
be needed. Same for run-time analysis. Different problem-domains they solve
for though.


If there is even an iota of agreement, wouldn't it be in the best interest
> of folks here to get vendors to ignore developer specific licensing and
> instead focus on enterprise concerns?



So some marketing guy one day grokked "there are only n-number of security
people per organization that scan run scanners, but there are Multiplier(n *
33.5) number of developers per organization....wow! We could sell all those
developers scanners!!! Ka-Ching."

That sort of thinking is pretty cool, leads to some cool sales growth graphs
and profitability projections. Need board-room meeting material, fits
perfectly into that circular-arrow graph everyone has with "TCO" and
"Lifecycle Management" and "ROI" and how it all loops together and saves
everyone big bags of money after they spend up front.

This circular "lifecycle-management" graph is labeled Security in the SDLC
and shows how you can buy a lot of developer/IDE-scanners and have even the
cheapest developers scan their code up front, and you'll save big in the
long run by avoiding those expensive security audits, compliance issues, and
having your smart (expensive) developers have to go back and fix all those
holes. Ka-Ching.

I haven't heard the business plans of any of the source-code-scanner vendors
in a year or two...but back then, I didn't hear one that aligned with any
enterprise software development or business plan reality that I have
experienced.

My thinking here could be off. I thought WAFs were a ridiculous idea but I'm
starting to see very effective use-cases for them, despite the ridiculous
claims and hyperbole that come from the WAF-vendor marketing nonsense.

Anyway, I think you are spot on James, and I think if you look at Gartner's
recent punditry you'll find some similar notions expressed.

Chow,

-- 
Arian Evans
software security stuff
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://krvw.com/pipermail/sc-l/attachments/20070607/1f260607/attachment.html