Dailydave mailing list archives
Re: Fuzzing
From: "Dustin D. Trammell" <dtrammell () tippingpoint com>
Date: Tue, 16 May 2006 10:04:15 -0500
On Mon, 2006-05-15 at 12:25 -0400, Jared DeMott wrote:
Both Miller and Jack mention randomness. But Jack mentions security. Peter also mentions security, but not randomness, he says invalid data. Some use the word fault injection or stress testing. Does the way we make data invalid HAVE to include randomness? Vaugnoux prefers a list of attacks rather than random data.
Depending on your level of knowledge of your target's input vectors and data structures, no, the invalid data does not have to be random. For example, if your fuzzing an application's protocol stack, and you know the message's first field in it's header is an 8 bit integer, why send it random data? You can fairly easily (and quickly) send it all 255 possible values for complete coverage. With a little bit of knowledge or educated guessing, you can vastly reduce the list of potential exceptional values you are sending to your target and thus greatly reduce your testing time.
It's clear that pure randomness is only good "one level deep". We need to add some structure (protocol knowledge) to advance multiple layers into any non-trivial protocol.
Correct. For a great example of this, try fuzzing the 3rd or 4th message in an IKE or IKEv2 session. Guess what? You might as well just implement the entire protocol and all the crypto required inside your fuzzer, because you're going to likely need it to get to the part of the session where you want to perform your fuzzing. (And on that note, YAY for racoon2 iked's -P option!)
Fuzzing is a testing technique (usually software) used to find bugs. That's about the only consensus currently.
I think you can also safely include here that fuzzing requires sending (hopefully) exceptional data to one or more of your target's input vectors.
So my question is to all you fuzzers and software testers: what is the difference between fuzzing and software testing? (If someone knows of a good software testing list, please forward this on.) In theory, they seem very similar. In practice, the second party, security focus of fuzzing has made it effective in finding exploitable bugs. But more academically, what is the difference between the two (or the definition of each).
Fuzzing is a subset of software testing. It just so happens that it's particularly useful testing method for security researchers...
The flip side of fuzzing (and testing) is determining when a fault has occurred. This seems to receive less attention than how input data is malformed. Here's an important question when defining fuzzing: What are the different (current and future) methods/trends for detecting when a failure/fault/bug has been found?
Summarizing your last two quotes (which I snipped for brevity), in the general sense, being able to observe your target's behavior in some form as you send it exceptional data is required. Sometimes this is achieved, using network protocol fuzzing as an example, by connecting to a service, sending exceptional data, then checking if the socket was forcibly closed or, if after disconnecting, attempting a legitimate reconnect. If the socket was forcibly closed or the reconnect fails, you've likely found a successful test case. In the case of non-network application testing, such as fuzzing an application's document parser, your input will likely be a malformed data file of some form, and the application is likely on your local machine. In this case you can attach a debugger, perhaps integrated directly with your fuzzer. If you're fuzzer/debugger combo tool then supported some form of scripting then you'd have a fairly robust and automated testing tool. -- Dustin D. Trammell VoIP Security Research TippingPoint, a division of 3Com
Attachment:
signature.asc
Description: This is a digitally signed message part
Current thread:
- Fuzzing Jared DeMott (May 15)
- Re: Fuzzing Matthew Franz (May 15)
- Re: Fuzzing Dustin D. Trammell (May 16)
- Re: Fuzzing Jared DeMott (May 16)