Re: Too Many Exceptions in the Firewall

[Michael Sinatra <michael () RANCID BERKELEY EDU>]

David Buckley wrote:
> Hello All,
>
>
>
> I would like to solicit the input of this list concerning some recent
> issues we are having with incoming faculty. We have recently hired some
> “high profile” faculty that was sought out by the administration to help
> compete on a national level.  The problem that we have is the moment the
> new faculty members arrive, they begin screaming because their systems
> under their desks are not accessible from outside and we are impeding
> their research. We have a perimeter firewall that does not except any
> inbound un-initiated requests. We attempt to offer centralized services
> for web hosting, database services, etc…

I believe in central services (economies of scale, professional
management, etc.), but I can also imagine a number of cases where
central services won't meet the needs of researchers.  Also, I don't
think the suggestions made by others on this list to allow access
through VPNs or only via ssh will work.  These researchers want to
present their work in a way that the public can access, usually via the
web and/or via java applets, but also via other protocols.

For example, imagine a professor who specializes in tele-robotics and
wants to have a website that allows anyone, anywhere to manipulate a set
of small robots around an enclosed space.  The professor does this as a
proof-of-concept for some research she is doing.  I have a hard time
imagining that the central web server will suffice for this project, nor
will requiring anyone who wants to participate in this project to use a
VPN client.

> The problem seems to be that
> the faculty wants to be able to touch the systems providing the hosting
> and be able to show off their quad-core Apple servers pulsing in their
> office.

That may be the reason, or it may be closer to the example I have given.
 Either way, I don't want to be in a position to judge whether a
professor's research is "good enough" to merit an exception to the way I
do things.

> They also go right to the top (CIO) and fuss causing him in turn
> to ask us to fix it immediately…therefore causing the firewall
> exception. Our worry is that this exception will soon be (or already is)
> out of hand and faculty will spread the word of these exceptions.  I know
> that not everyone supports perimeter firewalls

I think you have stumbled upon but one of the reasons some of us don't
like campus-border firewalls.

> but that has been our
> best solution for the time being considering man power/resources. Some
> questions I have on this are:

The answers that I am about to give cannot be understood outside of the
context of Berkeley's Minimum Standards policy (see
<http://security.berkeley.edu/MinStds/>).  Although the policy is
intended to be _minimum_ standards and not necessarily best practices or
some more detailed specifications for certain functions (mail servers,
web servers, etc.), it goes a long way toward improving the campus's
security posture because it empowers departments and users and forces
them to take responsibility for the systems that they attach to the
network.  It forces them to understand that the security of their
systems is in their hands and they cannot rely on some magic firewall in
the sky to protect them (it probably won't anyway).  I think that having
such a policy is the first step in distributing the costs of running
systems in a reasonably-secure manner to the people that are directly
deriving the benefits of the systems themselves and the freedom to do
what they want with those systems.

> How are you dealing with these issues? Do you have a policy that
> addresses this?

We block a very small number of ports at the campus border and we do not
provide exceptions.  We follow BCP 38, and we also block SNMP, TFTP and
some of the windows sharing ports (135-137, 138, 445).  The windows
stuff is not without controversy--there are people who want to see those
blocks removed and those who want to see them stay.

Beyond that, we provide three levels of firewall services to the campus:

1. Managed firewall services using either FWSM instances (like Georgia
Tech) or small Juniper firewalls where the network topology doesn't
allow the FWSM context to be used.  Very few departments have chosen this.

2. Same as #1, except the department manages the firewall or FWSM
instance.  This has been pretty popular and it has worked well for everyone.

3. Providing ports and appropriate vlan trickery to allow the department
to install and manage its own firewall, such as a Linux/netfilter or
OpenBSD/pf box.  A number of departments have chosen this route, mostly
before we were able to offer #1 and #2.

None of these options waives the Minimum Standards requirements that
hosts have host-based firewalls, and there are additional policies and
requirements that apply to systems that hold sensitive/restricted data.

For departmentally-managed firewalls, the department agrees to allow the
security group to audit configs, and also to allow security scans
through the firewall, from a designated set of machines belonging to our
security group.

> Do you have SLA’s that address this?

The purpose of an SLA is to provide refunds when service levels are not
met.  I am not sure where an SLA would come into play here.  Perhaps
there are operational consequences that might necessitate a refund of
the recharge money for the service.

> How do you reveal the responsibility for the data to the department?

I also don't understand this question/

> Has anyone delegated firewall exceptions to the discretion of the
> department? Does that work well?

Not the exceptions, but the firewall service.  It has worked very well
because it allows departments to be more restrictive for certain systems
than could be done at the campus border, it greatly narrows the
vulnerability perimeter, etc.  Even if we blocked everything at the
campus border, that's still 50,000 machines that could attack your
system, and with laptops constantly being brought to campus and taken
home (off-campus), there are plenty of opportunities for worms to get
inside the campus border even with blocks in place.  Pushing those
perimeters to the subnet level makes a lot of sense.

> What other protections do you have in place to augment the security for
> the exceptions?

Minimum Standards requires that all machines be patched, that they be
running modern OSes for which patches are being made available by the
vendor, that they run updated anti-virus, and that they have host-based
firewalls.  Obviously we can't expect 100% compliance, but the
compliance we have gotten has been a big help.

Like Georgia Tech, our security group is always scanning the campus IP
address space.  Scans are constantly looking for compromised or
vulnerable systems, especially for systems not patched against
recently-announced vulnerabilities.  Every quarter, our security group
produces vulnerability reports which go to each departmental contact,
and show all of the vulnerabilities on the systems that that department
manages.  It's not perfect, but it's still very helpful.

> Also, if anyone has transitioned from perimeter firewalls to a more
> layered approach, please describe your migration steps.

Like GA Tech, we never had a perimeter firewall, just minimal ACLs which
we still have.  The problem of managing exceptions, the impact on
network performance, the real mess it would create with respect to
troubleshooting (I speak from the experience of trying to troubleshoot
problems when traffic crosses departmental firewalls and/or when
researchers are trying to do high-performance stuff with other campuses
that do have border firewalls and I am called in to deal with
less-than-stellar performance) all contributed to this decision.


Some suggestions:

o I agree that you should create a policy.  Specifically create
guidelines for systems that will be exempted from the border firewall.

o Create a process by which faculty and others request exceptions.  They
should explain what exceptions they want and what they will do to
mitigate or manage the security risks.  They should also agree to follow
your "minimum guidelines" suggested above.  Exceptions should be
reviewed by your CIO, or better yet, by a small committee of
security-savvy folks taken from campus departments and the central IT
organization.  I think this should be possible given the size and
stature of Clemson.  This way, decisions made by the committee will have
 more legitimacy than if they were made by the "evil empire"--er, the
central IT organization.

The idea here is to provide more structure to the exception request than
mere jumping up and down at the CIO, to provide a paper trail, and to
provide some justifiable cost to the person who wants the exception.
They need to understand the issue enough and be willing to take a bit of
time to make a reasoned request--not just "click here for your exception."

By the way, are there really Apple servers that pulse under one's desk
when they're not behind a firewall?  I gotta get me one of those.

michael