Re: Practical Firewall Metrics

["Marcus J. Ranum" <mjr () nfr net>]

> What do the list feel about this - how do we set a criteria for selecting
> the best f/w, ID, etc for our secure networks - is it possible?

Obviously, from some things I've said in the past, I believe that it
is difficult and *MAY* be impossible.

There are 2 big problems we need to figure out how to overcome:
-> The products are end-user configurable
-> Vendors can make any claims they like

The first problem is serious because of the clue deficit in network
management today. The number of people who I'd say are qualified to
install a security policy in a firewall is a dwindling percentage
of the overall number of people installing firewalls. The prevalent
attitude seems to be "got firewall: am secure" -- even if the firewall
permits unsolicited incoming HTTP to backend systems.

How do we change this so that we can have some kind of auditable
configuration?? The answer is shockingly simple but it isn't likely
to happen. :) I'm going to be US-o-centric for a second but I suspect
this applies well enough to non-US.

In the US there are regulatory bodies that govern activities undertaken
by banks, brokerages, pharmaceutical companies, utilities, etc. Some
of those regulatory activities cover safety and technology safety.
Those regulatory bodies have COMPLETELY MISSED THE BOAT and decided
that the Internet isn't their problem. I believe they are wrong and,
as a taxpayer, I believe they are not doing thier jobs. Since they are
feds, of course, I can't fire them or stop paying my taxes. :( I
truly believe that the SEC (regulates stock market in US) should have
published an access control policy for brokerages connected to the
Internet. It should have been a high-level policy description ("service
X may be allowed between Internet and systems that are not connected
to trading floors...") written clearly enough that firewall vendors
could take that policy and code it into their products. Then when a
customer that falls under that regulatory package installs a firewall,
they could simply walk to the firewall's GUI, drag down a menu, and
tell it "apply the SEC brokerage Internet security policy."  This would
be very easy to do, very, very easy to audit, and folks who make
scanners like Ballista and ISS could very easily code validation
tests that regulators and auditors could apply to ensure conformance.
It ain't going to happen. Why? Because the regulators who own this
problem don't think they own this problem, and they think that having
an AOL account means they're on the Internet. The clue deficit I
mentioned in networking is even more severe in the regulatory agencies.
Even NIST and NSA have hopelessly dropped the ball on this one,
preferring to go down the common criteria "if we produce an unreadable
standard, no one will read it" rathole. :(

Network-1 makes a firewall called Firewall/Plus. It's a pretty good
firewall, but the one thing that I think is terrific about it is
that it has a bunch of policy templates for quick install. You
turn it on and tell it "install a highly paranoid access policy"
and it just does it. It has templates ranging from "academic open
access policy" to "closed except for Email" -- it's a terrific idea
and whenever I get a chance to browbeat firewall vendors (which is
less often, because I guess they are tired of me browbeating them!)
I tell them this would be a nice thing for them to support. It's
easy since it amounts to just a bunch of canned default configurations.

MAYBE it doesn't need to come from a regulatory agency. MAYBE it
could come from a security savvy third party (IETF, NCSA, or NSA, or
Price Waterhouse or The Pope or whatever) but it'd need to be
highly vendor neutral, clearly specified, and well marketed.

Need I mention that if such template standards existed, they
would form useful backbones for IDS rule-sets, network scanners,
and compliance audit tools? One of the problems with IDS is that
it's hard to define "normal" -- having a templated policy defines
a baseline of "normal" in a way that would be highly useful. If
you tell me you are running under "SEC brokerage template" and I
know thereby that there should be no incoming telnet EVER then
it's trivial for me to set up an NFR to tweet a whistle when it
sees incoming telnet.

Summary: it's easy to do this, technically, but it'd take
marketing, educating customers and regulatory agencies, and
it'd be a Big Pain In The Butt for someone to do. The vendors
won't do it because they are too busy making money.

The second issue is that validating firewalls is EXTREMELY hard
because vendors can make whatever ridiculous claims they like
and get away with it. "Our new turbo-whomping voodoo packet
screening meta-analysis layer detects 99% of the attacks it
knows how to detect!!!" NOBODY is able to stand up loudly and
declare "BULLS**T!"   The paper Christopher alluded to was my
attempt to do so, and it was well received in very narrow
circles but made little headway with its target audience (journalists)
who I was trying to educate about the perils of well-marketed
"certification organizations" for firewalls. There are 3 reasons
that vendor claims are so ridiculous:
1) since the units are end customer installable, any error is
        the customer's fault, QED
2) journalists are generally too busy and don't have access to
        sufficient expertise to torture test products
3) regulatory agencies (see screed above) never set any useful
        standards so there are no useful vocabularies in which to
        discuss technologies compliance

Again, there is a solution to this problem which is very
simple but ain't gonna happen. Some neutral third party has to
do a completely pro bono or customer funded equivalent of
computer security Consumer's Reports. There has to be no
money taken from vendors. There has to be no advertising.
There has to be highly trained technical staff, good writers,
and editors that let them say "product XYZ is BULLS**T and
we tested it and here's why."

The reason this ain't gonna happen is because the caliber
of experts needed to do the reviews is *expensive* and they
tend to be tough people to work with because they are all
too busy for that kind of thing. :) There was a time a few
years ago when I thought I was going to get rich (I was, for
about a month before the market corrected me back to a
negative number) when I thought that it'd be fun to retire
and attempt to organize such a thing pro bono. I doubt that
enough customers would pay for such a thing to make it
survivable. POSSIBLY one of the analyst firms like Gartner
Group could get away with it but the profit margins on
testing are not as high as they are on making prognostications
about where the industry is going.

Anyhow - sorry about the rant.

Summary:
        Can it be done? Yes.
        Will it be done? Not likely.

mjr.
--
Marcus J. Ranum, CEO, Network Flight Recorder, Inc.
work - http://www.nfr.net
home - http://www.clark.net/pub/mjr