WebApp Sec mailing list archives
Re: Solutions, Results, and Comments - Was [ISA Server and SQL Injection]
From: Jeremiah Grossman <jeremiah () whitehatsec com>
Date: Mon, 28 Feb 2005 08:28:31 -0800
On Wednesday, February 23, 2005, at 09:36 PM, Jeff Williams wrote:
There are essentially four ways of finding flaws in an application: - Vulnerability scanning with signatures - Manual penetration testing - Static analysis of source or binary - Manual code review
In your description, aside from automation, what is the difference between bullet 1 and bullet 2? I've view the testing process as 3 fundamental methods. Automated "scanners" can be applied to each method, but essentially they are performing the same process we'd do by hand.
- black-box testing (functional testing) - static binary analysis - source code review
Any application review that relies on a single technique is going to be less cost-effective than one that uses a combination. And when I say "cost effective" I mean more serious flaws found for the $. The 80-20 rule is nice, but you've got to find the right 80%. Finding 4000 minor flaws and missing a major obvious hole doesn't help anything.
"Find the right 80%". This is really good way to look at it.
Just so I'm totally clear here -- no matter how much you're planning to spend on finding vulnerabilities in your application, you're better off including some scanning, some code review, some static analysis, and some penetration testing. Skilled analysts with a full toolbox of techniques is what produces the best bang for the buck.
Well said and I think people are leaning that direction. The challenge customers face is how much of what method to use and when. This answer is not readily apparent and certainly not agreed upon in the industry. We've seen people on the list comment on how they do (or have done) secure software in the past. A process that made sense to them. It always appears to be a combination... but the way the process is implemented is always a bit different. This an area within the industry I see lacking good information. Assisting customers to find the right balance.
Also, I haven't come across much of anyone doing scans on binaries, but I am in the web app sec world)
Anyone claiming that a single technique can find nearly all (if not all)vulnerabilities or costs less than another technique just doesn't get it. Savvy consumers will look for reviewers that use best of breed scanning and static analysis tools. They'll also look for serious pentest and code review expertise with the technologies they're using.
One testing method might actually find all the vulnerabilities (bugs), but its not something you want to rely upon. The premise is theoretically code can free of bugs, we just have no way to prove it. Standard Turing logic. Similarly to the way we diversify our investment portfolio to provide safe and consistent results. Its no different with security, we should diversify our solutions and go for defense in depth.
Regards, Jeremiah-
Current thread:
- Re: Solutions, Results, and Comments - Was [ISA Server and SQL Injection], (continued)
- Re: Solutions, Results, and Comments - Was [ISA Server and SQL Injection] David (Feb 23)
- Re: Solutions, Results, and Comments - Was [ISA Server and SQL Injection] Jeremiah Grossman (Feb 28)
- storing SSNs, CCNs, password in the DB Francesco (Feb 28)
- Re: storing SSNs, CCNs, password in the DB Adam Shostack (Feb 28)
- Re: storing SSNs, CCNs, password in the DB Francesco (Feb 28)
- Re: storing SSNs, CCNs, password in the DB Andrew van der Stock (Mar 01)
- Re: storing SSNs, CCNs, password in the DB Paul Johnston (Mar 01)
- Re: storing SSNs, CCNs, password in the DB Joseph Miller (Mar 01)
- Re: storing SSNs, CCNs, password in the DB Alvin Oga (Mar 01)
- Re: Solutions, Results, and Comments - Was [ISA Server and SQL Injection] Jeff Williams (Feb 28)
- Re: Solutions, Results, and Comments - Was [ISA Server and SQL Injection] Jeremiah Grossman (Mar 01)
- Re: Solutions, Results, and Comments - Was [ISA Server and SQL Injection] Jeff Williams (Mar 01)
- Re: Solutions, Results, and Comments - Was [ISA Server and SQL Injection] Jeremiah Grossman (Mar 01)
- Re: Solutions, Results, and Comments - Was [ISA Server and SQL Injection] Jeff Williams (Mar 01)
- Re: ISA Server and SQL Injection Paul Johnston (Feb 23)
- RE: ISA Server and SQL Injection Mark Curphey (Feb 23)
- Re: ISA Server and SQL Injection Paul Johnston (Feb 23)
- RE: ISA Server and SQL Injection Mark Curphey (Feb 23)
- Re: ISA Server and SQL Injection Paul Johnston (Feb 28)
- Re: ISA Server and SQL Injection Stephen de Vries (Feb 28)
- Re: ISA Server and SQL Injection Jan P. Monsch (Mar 01)