Re: Solutions, Results, and Comments - Was [ISA Server and SQL Injection]

["Jeff Williams" <jeff.williams () aspectsecurity com>]

> The particular web site itself is more the cause of the discrepancy than 
> anything else. What is clear though is you have to test
> using both methods no matter what.

Agreed. Actually all four methods (scan, pentest, static analysis, code 
review)

> > If you are in the web app sec world and you aren't
> > scanning binaries,  you're wasting your customer's
> > time and money.
>
> This is a bit strong and largely subjective without describing what is 
> supposed achieved using the results. In every engagement, whether your an 
> attacker or a tester, you not always going to have access to the source 
> code or binary. This is the framework your given to conduct your work.

Good point. I should clarify. If your customer won't let the security 
analysts analyze the code, then *they* are wasting their time and money --  
at least if the goal is to find the most serious security flaws in the most 
cost-effective manner.  If they're looking to "check the box" or have some 
political goal for the review, then perhaps not finding flaws efficiently is 
what you want.

It's not my experience that there is no flexibility in how these assessments 
are done. I think we have an obligation to help companies understand the 
weaknesses with doing only scanning or only pentesting. If you think a 
combination of techniques IS more cost-effective, then you should be 
explaining to them that using a combined approach yields more complete, more 
accurate results for the dollar.

> 1) What exactly is scanning strategy they employe to find these issues? 
> I've had this half answered.  I could be wrong, but these tools appear try 
> to find areas of input. Then decide if the data has been properly 
> sanitized. Looking at the results they appear to point you to the area 
> where a potential issue might be, but fall short of certainty. Is this 
> your impression?

Data-flow analysis is only one part of static analysis. These tools also 
look at where security mechanisms are invoked to ensure that they're being 
used properly. They look for coding patterns that generally indicate a 
security vulnerability.  And they look at the structure of the code, to see 
if security mechanisms are properly isolated and centralized. They 
definitely fall short of certainty -- but they're very good at steering an 
analyst to the right parts of the code to look at.  You should make sure it 
covers the technology you're using (especially extra libraries) and that 
it's easy to train it to know about the custom security libraries involved 
in the application.

> 2) How well have have performed in practice on real production web 
> application code? Have you used many? What were the results like?

Our first J2EE static analysis tool was built in 1999, and we've been using 
them internally ever since. Recently we've been using some of the 
commercially available tools as well. Our results have been excellent --  
especially in combination with the other approaches. They are very good for 
security analysts, as they can quickly point to sections of a large software 
baseline that need manual review. For example, reviewing an application for 
SQL injection problems is just a matter of clicking through the database 
interactions to be sure they don't include unvalidated user input. In many 
cases this will be faster than scanning, and will almost certainly be more 
accurate.

--Jeff

Jeff Williams
Aspect Security, Inc.
http://www.aspectsecurity.com