Re: High Speed Firewalls

["Paul D. Robertson" <proberts () clark net>]

On Mon, 6 Mar 2000, Bennett Todd wrote:

> Correct me if I'm mis-reading you here, but as far as I can tell,
> you didn't like an early release of Distributed Director, and so
> condemned LocalDirector as well.

We had both in conjunction (DD to pick the site, LD onsite), and my main
query was to discover in what conditions a non-distributed site would be
important architecturally so that the DD component wouldn't be ncessary
(since I'm more willing to write that part off unmercifully.)

I've always considered "local balancing only" half the solution, so I'm
particularly interested in cases where it's architecturally a goal rather
than giving up ground to the money gods of multiple sites.

> Perhaps you don't have any use for a simple local load balancer.
> I've found them very useful. I'll happily use a pair of
> LocalDirectors in H-A mode to scale a farm up to many, many times
> the capacity of any single component member.

BigIP does this as well, and seems to be the "industry standard", which is
why I wanted to know if you'd had a chance to contrast the two.  You
normally don't strike me as vehminantly rabid for a particular product, so
I was wondering how much was "just works for me" versus "something else
didn't work."

> For many applications, I'll prefer to place the job of distributing
> the traffic about the internet contractually in the hands of the
> provider. If they are doing their job well, they'll be able to carry
> the traffic out to their borders.

I've never believed in single points of failure, be they companies, ASNs,
networks or routing policies.

> I've never tried out Distributed Director. If I wanted wide-area
> load balancing, I'd try out various alternatives, but I wouldn't
> approach the problem expecting any of the alternatives to work much
> better than a round-robin DNS hack.

They do, especially for high-volume sites.  There is associated traffic
from the load balancer's distributed component that freaks out the
occasional firewall admin, especially if it's being aggressive at nearest
colo direction, but they're quite effective compared to RR even with the
random RR stuff in BIND8.

I've always thought it'd be better to multi-thread the resolver component
and force a CNAME lookup over each site prior to picking the nearest site,
but I suppose the initial latency would suck pretty badly (but I hate to
see people's alarms going off for no good reason, and I hated even more
having to explain load balances to paranoid sleepless firewall admins.) 

> There's a very big difference here. LocalDirector has a job assigned 
> to it that can be done, and can be done well. It brings passive
> performance tracking to the party, that's its big improvement over
> its competition.

So, it's your assertion that passively tracking server performance is a
significant improvement over actively doing so?  

> Distributed Director can't in principle do its job as well. Where
> the LocalDirector's job is to direct traffic to the
> currently-fastest server in a farm, from the point-of-view of the
> choke point where the farm hits the net, the job you'd like a
> Distributed Director to perform is assigning customers to the farm
> nearest them --- but there's no way to do that without non-standard
> and explicit assistence from the application.

I'm less-concerned with nearest farm (though there are business-side
people who should be concerned with that) than I am with site DOS or
natural failure modes.  For that, most of the solutions work well enough,
including my little "make something authoritative and let BIND do the
rest" hacks.

At my last job, our most visisble Web site was in the "over a hundred
million hits a day" range at peak load during significant events, and
while I wasn't directly assigned to that business unit, I did a fair
ammount of work with them on issues such as this.  I've also worked with
single-site Web hosting providers on the same set of issues (where my
recommendation for multiple colo facilies weren't budgetable), and done
evals of most of the major colocation facilities in the Washington D.C.
area from around the MAE out.  I've been surprised at the number of
glowing little F5 logos I've seen in cages, and considered it indicitive
of the initial experiences we had with the Cisco products.  If that's not
true (and it's probably an invalid assumption- hence my queries here) then
I'd like to figure out why F5 is sitting on top of almost every 75xx or
55xx piece of Cisco gear I've seen when I've been looking around (or
pointed not looking around in the case of those colos which discourage
such looking ;)) a significant number of cages.  More pointedly, I could
probably count the number of LD boxes I've seen at colos, and there's no
way I could count the number of F5 boxes, and again the colo people seem
to offer F5 managed services almost exclusively (or that's my impression-
again it could be wrong.)   

If you could provide some scale and architectural background, I'd like to
put the DD/LD combo back on my list of things to look at when I get a
chance, but I've not seen BigIP fail under load if the number of BigIP
boxes is correct for the site- I've seen the DD/LD combo hose back when
the site in question was only at about 35M hits/day (once again, my memory
is fuzzy, but that's the best of my recollection and I'm perfectly willing
to admit the parts I don't recall well.)

Paul
-----------------------------------------------------------------------------
Paul D. Robertson      "My statements in this message are personal opinions
proberts () clark net      which may have no basis whatsoever in fact."
                                                                     PSB#9280

Attachment: _bin
Description: