Re: Solutions, Results, and Comments - Was [ISA Server and SQL Injection]

[Jeremiah Grossman <jeremiah () whitehatsec com>]

On Monday, February 28, 2005, at 09:12  AM, Jeff Williams wrote:

> > Automated "scanners" can be applied to each method, but essentially 
> > they are performing the same
> > process we'd do by hand.
>
> Scanners *can't* perform "the same process we'd do by hand."  This is 
> because a skilled penetration tester can learn how an application 
> works and adjust.  Scanners can't.  The vast majority of serious (i.e. 
> high risk) problems we've found over the past 7 years could *never* 
> have been identified by a scanner, even though many were fairly 
> obvious to a human tester.

I'm well aware of the limitations of scanning and agree with your 
assessment.  I've been giving presentations expressing this topic for 
some time now.  I should have said scanners do "some" of the things we 
traditionally do by hand, lest I be mistaken for saying they do all 
things. Personally I'm not sure about the numbers of high severity 
issues yet and how they are "mostly" found.  We've identified many 
high-severity issues using automation and many using contextual logic. 
The particular web site itself is more the cause of the discrepancy 
than anything else. What is clear though is you have to test using both 
methods no matter what.


> > Also, I haven't come across much of anyone doing
> > scans on binaries, but I am in the web app sec world)
>
> If you are in the web app sec world and you aren't scanning binaries, 
> you're wasting your customer's time and money.

This is a bit strong and largely subjective without describing what is 
supposed achieved using the results. In every engagement, whether your 
an attacker or a tester, you not always going to have access to the 
source code or binary. This is the framework your given to conduct your 
work. Also, are large portion of web applications are script-based and 
don't have binaries to analyze anyway.


> Scanning binaries is (in some ways) easier than scanning source code 
> because the compiler has done much of the difficult parsing work 
> already -- meaning you don't have to replicate the customer's build 
> environment.  For example, scanning binaries will allow you to find 
> potential SQL injection points (and more importantly, non-SQL 
> injection points) in seconds. You'll immediately see whether the web 
> application uses dangerous libraries and methods. You're missing out 
> on an important source of information.

This is a good topic and I've been asking around about this frequently. 
Source code scanning to find web application security vulnerabilities 
(SQL INJ, XSS). The theory described in the literature seems promising 
and compelling from a results standpoint, which is why I find it 
interesting.  I still a couple of questions though that you as a 
practitioner might be a share some thoughts on.

1) What exactly is scanning strategy they employe to find these issues? 
I've had this half answered.  I could be wrong, but these tools appear 
try to find areas of input. Then decide if the data has been properly 
sanitized. Looking at the results they appear to point you to the area 
where a potential issue might be, but fall short of certainty. Is this 
your impression?

2) How well have have performed in practice on real production web 
application code? Have you used many? What were the results like?


Regards,

Jeremiah-