Full Disclosure mailing list archives
Re: How secure is software X?
From: "David Litchfield" <davidl () ngssoftware com>
Date: Sat, 13 May 2006 12:17:31 +0100
Hi Justin,
One thing you have to keep in mind is that a lot of things are incredibly variable when dealing with this subject. For instance, suppose you want to ensure that the URI in a web server is not overflowable. So you test with something like GET /[AAAAAAAAA x 4096] HTTP/1.1 Host: foobar.com Connection: closeThis is all fine and well, unless at 8192 is where the overflow takes place,or if I can stick as many characters as I want in, so long as I am using HTTP 1.1 and not HTTP 0.9, or if I am using HTTP/1.1 and Host doesn't contain a 36 backslashes, et cetera.This is generally why fuzzing is mostly inconclusive because it often missesa lot of conditions and you have essentially assured nothing. Without in-depth analysis of each software package you are basically pushing snake oil. There are just far too many variables to really standardize such a thing.
There are a few things to remember:1) There are still too many products that fall to simple fuzzing. Having a standard that employs fuzzing as part of it means that (hopefully!) vendors will develop at least to that level - this raises the bar so to speak. 2) Not all fuzzers are born equal. Having written a fair few in my time I do realize that condition based fuzzing is important. A very simple but quite common example, to add to the ones you given, is with SMTP fuzzing. Some overflows only trigger after an EHLO greeting but not after a HELO. A good fuzzer and a good fuzzing process should take into consideration as many conditions as possible. 3) Fuzzing would only be part of the standard to be proposed. There are code/assembly scanning tools which can be incorporated amongst other things.
Cheers, David
Best Regards, Justin Ferguson Reverse Engineer NNSA IARC 702.942.2539 "It is a capital mistake to theorize before one has data. Insensibly onebegins to twist facts to suit theories, instead of theories to suit facts."-- Sir Arthur Conan Doyle-----Original Message----- From: Adam Shostack [mailto:adam () homeport org] Sent: Friday, May 12, 2006 11:35 AM To: David Litchfield Cc: bugtraq () securityfocus com; full-disclosure () lists grok org uk; ntbugtraq () listserv ntbugtraq com; dbsec () freelists org Subject: Re: How secure is software X? Hi David, Very briefly because I'm swamped today: Please consider bringing some of this to Metricon (https://securitymetrics.org/content/Wiki.jsp?page=Welcome) Also there's a project of US DHS/NIST and probably others called SAMATE Software Assurance Metrics and Tool Evaluation http://samate.nist.gov/index.php/Main_Page which might be of interest. Adam On Fri, May 12, 2006 at 02:59:17AM +0100, David Litchfield wrote: | How secure is software X? | | At least as secure as Vulnerability Assessment Assurance Level P; or Q | or | R. Well, that's what I think we should be able to say. What we need is an | open standard, that has been agreed upon by recognized experts, against | which the absence of software security vulnerability can be measured - | something which improves upon the failings of the Common Criteria. Let's | choose web server software as an example. When looking for flaws in a new | piece of web server software there are a bunch of well known checks that | one would throw at it first. Try directory traversal attacks and the | several variations. Try overflowing the request method, the URI, the query | string, the host header field and so on. Try cross site scripting attacks | in server error pages and file not found messages. As I said, there's a | bunch of checks and I've mentioned but a few. If these were all written | down and labelled with as a "standard" then one could say that web server | software X is at least as secure as the standard - providing of course the | server stands up. | | For products that are based upon RFCs it would be trivial to write a | simple | criteria that tests every aspect of the software as per the RFCs. This | would be called Vulnerability Assessment Assurance Level: Protocol. If a | bit of software was accredited at VAAL:Protocol then it would given a | level of assurance that it at least stood up to those attacks. | | Not all products are RFC compliant however. Sticking with web servers, | one | bit of software might have a bespoke request method of "FOOBAR". This opens | up a whole new attack surface that's not covered by the VAAL:Protocol | standard. There are two aspects to this. Anyone with a firewall capable of | blocking non-RFC compliant requests could configure it to do so - thus | closing off the attack surface - from the outside at least. As far as the | standards go however - you'd have to introduce criteria to cover that | specific functionality. And what about different application environments | running on top of the web server? And what about more complex products such | as database servers? I suppose at a minimum for DB software you could at | least have a standard that simply checks if the server falls to a long | username or password buffer overflow attempt and then fuzz SQL-92 language | elements. It certainly makes standardization much more difficult but I | think by no means impossible. | | Clearly, what is _easy_ is writing and agreeing upon a VAAL:Protocol | standard for many different types of servers. You could then be assured | that any server that passes is at least as secure as VAAL:Protocol and for | those looking for more "comfort" then they can at least block non-RFC | compliant traffic. | | Having had a chat with Steve Christey about this earlier today I know | there | are other people thinking along the same lines and I bet there are more | projects out there being worked on that are attempting to achieve the same | thing. If anyone is currently working on this stuff or would like to get | involved in thrashing out some ideas then please mail me - I'd love to hear | from you. | | Cheers, | David Litchfield | http://www.databasesecurity.com/ | http://www.ngssoftware.com/
_______________________________________________ Full-Disclosure - We believe in it. Charter: http://lists.grok.org.uk/full-disclosure-charter.html Hosted and sponsored by Secunia - http://secunia.com/
Current thread:
- Re: How secure is software X?, (continued)
- Re: How secure is software X? Brian Eaton (May 12)
- Re: How secure is software X? sebastian . rother (May 12)
- Re: How secure is software X? Lucien Fransman (May 12)
- Re: How secure is software X? Lucien Fransman (May 12)
- Re: How secure is software X? Roman Medina-Heigl Hernandez (May 13)
- Re: How secure is software X? Valdis . Kletnieks (May 13)
- Re: How secure is software X? David Litchfield (May 13)
- Re: How secure is software X? Mike Hoskins (May 13)