Re: Static Analysis part 5

["Rafal M. Los" <rafal () ishackingyou com>]

Halvar, all...

    I think it's even more complex than that.  The whole problem with static 
analysis is the impact of the false-positive.  We take for granted that 
Static Analysis generally makes piles of false-positives through analysis 
(either you're too strict, or too loose... either way is bad) and what that 
does goes deeper than just make that particular set of results questionable. 
Infosec generally has a hard enough time convincing developers (this is from 
experience, not anything else) that they have issues... but then dropping a 
300-page report wherein you have thousands of "possible" issues (whether you 
use a probability scale or not) means that it's going to degrade further the 
*reputation* that the InfoSec folks have with the rest of the development 
organization.  Vetting false-positives often requires a developer (or at 
least a compiled app to test with) meaning that there is a much deeper 
involvement to "get at the truth".

  Then there's the issue of customer-supplied sanitization and validation 
routines.  Tainted data is one thing but how does a static analysis engine 
determine that data that passes through Method_X(), for example, doesn't 
have the right level of sanitization or validation?  You simply can't 
account for all the creative ways that developers can come up with to scrub 
var's, and it gets worse...

  Don't get me wrong, I think there is a place for static analysis - but the 
truth is that there are limited use-cases for the technology.  Halvar's 
examples illustrate a giant gaping hole in static analysis, being able to 
"be certain" is another huge problem... what you end up with is a report 
that requires a lot of further analysis, extensive vetting and a (potential) 
reputational hit to the InfoSec folks... that's a cost/benefit analysis that 
tips heavily towards the cost.

__
Rafal M. Los
Security & IT Risk Strategist

 - Blog:                http://preachsecurity.blogspot.com
 - LinkedIn:    http://www.linkedin.com/in/rmlos
 - Twitter:     http://twitter.com/RafalLos

--------------------------------------------------
From: "Halvar Flake" <halvar () gmx de>
Sent: Monday, July 27, 2009 10:40 PM
To: "dave" <dave () immunityinc com>
Cc: <dailydave () lists immunityinc com>
Subject: Re: [Dailydave] Static Analysis part 5

Tainting / untainting is generally a difficult topic.

- If I calculate the length of a tainted string, is the result tainted ?
- If I check some boundaries on this string, is the result untainted ?
- What if the boundaries I checked for are wrong ?

Can of worms, difficult to solve.

Cheers,
Halvar

dave wrote:
> If you haven't read it yet (and you like to read things), this is worth
> reading:
>
> http://samate.nist.gov/docs/NIST_Special_Publication_500-279.pdf
>
> There's a lot of meta-questions here, like "What sorts of
> vulnerabilities can automated techniques really find?" Scalability is
> hard too. "Can we look at 4500 buffer overflow bugs to triage them?" the
> paper asks.
>
> Many people don't see why it's hard to do static analysis automatically,
> which I think is always worth thinking about.
>
> For example, recently I was at a conference looking at a vendor's
> implementation of the traditional "data-flow-then-alert-on-tainted-data"
> algorithm. (As opposed to the
> "solve-massive-equation-to-understand-data-constraints" style).[1]
>
> We were looking at PHP, and I couldn't for the life of me figure out why
> the system was alerting on a particular code path. I will reproduce it
> here in python, since my PHP is weak.
>
> d={} //DEFINE dictionary
> d["HI"]=TAINTED_USER_DATA; //insert some tainted data to it, whole
> dictionary is tainted now.
> #stuff goes here that is not relevant (we think)
> os.system(d["BYE"]) //ALERT!
>
> It turns out the analysis tool tainted the whole dictionary if any
> tainted data was inserted into it, because all the other solutions for
> that are...insanely complex. Maybe one of the static analysis guru's on
> the list knows of the best way to solve that particular problem and
> wants to share. (Or maybe there isn't a solution for that problem, and
> no one wants to point that out either? :>)
>
> -dave
> [1] Scanner Pong's rules are as follows. You must use the test programs
> the vendor brings with them to demonstrate their static analysis tool.
> For each vulnerability you look at you lose 1 point. For each
> vulnerability you look at that you show to be a false positive, you get
> 2 points. For each vulnerability you look at where the tool has missed
> an interesting vulnerability within 30 lines of code you get 5 points.
_______________________________________________
Dailydave mailing list
Dailydave () lists immunitysec com
http://lists.immunitysec.com/mailman/listinfo/dailydave


_______________________________________________
Dailydave mailing list
Dailydave () lists immunitysec com
http://lists.immunitysec.com/mailman/listinfo/dailydave 

_______________________________________________
Dailydave mailing list
Dailydave () lists immunitysec com
http://lists.immunitysec.com/mailman/listinfo/dailydave