Re: Firewall bake-off?

["Marcus J. Ranum" <mjr () ranum com>]

K K wrote:
Same goes for just about every other "proxy" or "deep inspection"
> product on the market -- some protocols are deep, others shallow.
> If you're lucky, the vendor clearly indicates which is which.

Yup!

The problem (and you know this) is that throughput is not the
important property of a firewall. Firewalls, after all, are security
devices, not bandwidth managers. Certainly they affect throughput
and that's a consideration, but the industry has been lazy in
using performance as the distinguishing measure of a firewall
for the last 15 years. I've always interpreted that as a sign of
the industry's willingness to prey upon the uninformed
consumer more than anything else.

When Peter Tippett and Bob Bales started up ICSA Labs (back
in the day, as it were...) we had a lively discussion surrounding
the whole topic of "how to test a firewall for quality" (rather
than throughput) and came up dry. The result was ICSA's approach,
including elements of my suggestion that vendors be required to
document aspects of the security properties of their products.
At least it would bring security into the discussion about measuring
firewalls, instead of just packets per second or mega-URLs or
some other ludicrousness.

As much as I hate simple "signature count" as a metric, that'd be
the best metric I could think of nowadays. I'd ask the vendor to
document how many direct match security condition checks and
how many indirect match checks were in the data pipe on a
per-protocol basis. By "direct match" checks I mean a simple IDS-style
signature or code-rule such as:
if ( proto == http && payload >< "POST phf.pl" ) ...
if ( proto == ftp-cmd && payload >< "^USER" && strlen(payload) > 156) ...
and by "indirect match" I mean checks in which the protocol is
positively outlined and deviations from the positive outline are
blocked. Indirect match rules would, obviously, be far fewer - but
a single indirect match rule would be likely to offer much more
protection than a slough of direct matches. It might also help
to count in terms of whitelisting check points and blacklisting
check points. Maybe then it'd be worth talking about throughput
on a per-protocol basis with the check sets turned on, and turned
off.

That way I could look at a product and go "Hm, this is just an antivirus
toy pretending to be a firewall... It 'knows' 5 signatures for HTTP
vulns and other than that it's shuffling packets. No thanks!" I'm absolutely
certain that if most customers actually understood the security
check-sets of their products they'd be horrified to see that most
of them check:
per-protocol: ip source, ip destination
and that's it. Yet the same customers will say (with a straight
face) that IOS router ACLs "are not a firewall" or that "my company
doesn't allow IPtables as a replacement for a firewall."

This industry is really sad, sometimes.

mjr. 

_______________________________________________
firewall-wizards mailing list
firewall-wizards () listserv icsalabs com
https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards