Dailydave mailing list archives
Re: [fuzzing] Coverage and a recent paper by L. Suto
From: "J.M. Seitz" <lists () bughunter ca>
Date: Mon, 29 Oct 2007 12:57:41 -0800
Andre,
Developers can also improve automation through continuous integration, automated static code analysis security scanners, automated model-checkers, and build-integrated system tests (e.g. using Canoo WebTest, Jameleon, staf.sf.net, etc).
Continuous integration is a definite plus, if you can couple a per-build smoke test with a nightly build robustness test (fuzzing), you are sitting pretty. And generally it's a great first warning that something is up. The real problem is in designing an automation framework that is extensible enough to not only allow custom fuzzers, but is able to adapt to changes in the test targets, network parameters, protocol/file format changes, etc.
Design and code review is difficult to automate, but we can "automate" processes through workflow tools. Blueinfy has some free tools that help with automating aspects of code review, and Atlassian has a product called Crucible to improve code review workflow. Atlassian and ThoughtWorks/OpenQA are great resources for continuous integration developer testing improvements.
Atlassian's JIRA also has some excellent workflow tools that can extend into any one of their products and SVN, etc. It is easily automatable through build tools like Tinderbox, Hudson, etc.
More advanced operations/maintenance security testing can be done using EFS and/or WI/AppScan/Hailstorm against weak areas of code coverage (or areas which represent more complexity) before adversaries do the same. I wouldn't say that these solutions are "custom" built in the same way as complete protocol dissection, reverse engineering, or other generated test scenarios. However, each of these tools does require some "configuration", usually by an expert.
How do you know where your weak areas of code are? Is EFS going to successfully dissect a binary protocol and give you metrics that are useable? Not really. In my experience, and I love Jared like a brother, EFS works very well against ASCII protocols, with a known set of seeds. If you turn it loose against an unknown binary protocol, it doesn't do so well. This is where we need to advance the whole notion of automated binary protocol dissection, and then couple that will a block-based (or genetic if you want) fuzzer that can then utilize the results that your dissection yielded. Look for something like this in December, by some goofy looking Canadian dude :)
Unfortunately, the largest problem we're facing with the above strategies is the growing rate of code vs. the growing rate of qualified experts. Make sure to check out Felix Linder (FX)'s talk from HITB 2007, available here - http://conference.hackinthebox.org/hitbsecconf2007kl/materials /D2T1%20-%20Felix%20Lindner%20-%20%20%09%20Attack%20Surface%20 of%20Modern%20Applications.pdf
You got that right, in a development shop you are lucky to have one person in the whole company that can actually test the corner cases, use a debugger to track down problems (without source), demonstrate attack surface, yadayada. Problem is there are all kinds of jobs for security engineers, pentesters, app-pen testers, etc. But how often do you see a job posting like: "Hax0r In the QA Department Needed". That's where the demand _has_ to start moving towards. And please folks, pay your QA hackers well :) JS _______________________________________________ Dailydave mailing list Dailydave () lists immunitysec com http://lists.immunitysec.com/mailman/listinfo/dailydave
Current thread:
- Re: Coverage and a recent paper by L. Suto, (continued)
- Re: Coverage and a recent paper by L. Suto Charles Miller (Oct 16)
- Message not available
- Message not available
- Re: [fuzzing] Coverage and a recent paper by L. Suto matthew wollenweber (Oct 17)
- Re: [fuzzing] Coverage and a recent paper by L. Suto JFV (Oct 18)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Nicolas RUFF (Oct 25)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Alexander Sotirov (Oct 26)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Lance M. Havok (Oct 26)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Nicolas RUFF (Oct 27)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Lance M. Havok (Oct 27)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Alexander Sotirov (Oct 28)
- Re: [fuzzing] Coverage and a recent paper by L. Suto Andre Gironda (Oct 29)
- Re: [fuzzing] Coverage and a recent paper by L. Suto J.M. Seitz (Oct 29)
- Re: Coverage and a recent paper by L. Suto Charles Miller (Oct 16)
- Re: Coverage and a recent paper by L. Suto J.M. Seitz (Oct 29)