Re: High Speed Firewalls

[Eric Hall <firewall-wizards () darkart com>]

At 12:34 -0500 3/6/00, Mike Barkett wrote:
> On Fri, 3 Mar 2000, Bennett Todd wrote:
[snip]

> BT>
> BT>As far as I know, the Cisco LocalDirector remains unique among load
> BT>balancers in the basic way it works.
> BT>
> BT>It dispatches incoming requests to servers in the farm, and keeps
> BT>a notepad to make the assignments "sticky"; so far they're all the
> BT>same. But LocalDirector keeps track of how quickly each server in
> BT>the farm responds to a request, and always assigns the next new
> BT>connection to the server who responded fastest. This allows it to
> BT>automatically drop failed boxes out of the pool, and re-introduce
> BT>them when they're brought back (HA failover); again, all the
> BT>load balancers should be able to do that. But LocalDirector also
>
> Sorry, in my opinion, this "feature" is a bug, when put into practice.
> Imagine the scenario in which a web server has failed, and a 404 error
> comes up for the main page.  This server will be much quicker to respond
> than the full e-commerce/img/java-encrusted blicki.  LD starts sending
> more and more requests to the failed server, and you've got a bad
> situation on your hands.  I have seen it happen in extremely high-volume
> e-commerce environments and it's not pretty.
>
> Hopefully Cisco has fixed or will fix this problem, but even if they did,
> the LD would not be the superior product.  You can set the Alteons to
> expect a certain string of HTML code, and regularly monitor that type
> connection at layer 4.  Now, that doesn't entirely make up for Alteon's
> lackluster NAT support, but that type of monitoring is where Cisco wants
> to be with their product.
>
> -MAB

	Foundry ServerIrons do the same sort of thing as the Alteon (from 
the above description) - make an HTTP request and look for a certain 
string in the returned HTML.  I like that a lot, you can look through 
the web server to a backend DB or such if you like.
	Note that it has one 'interesting' human failure - if you use a 
simple page for the health check ('cause the checks happen quite 
often) and that page gets blown away from your servers the redirector 
takes them all out of service.  Fairly easy to do if there's any sort 
of disconnect between the network people and the content folks 
(<sarcasm> but of course that never happens </sarcasm>.
	I think the Foundry gear will also notice which servers have the 
least number of current connections and favor them for new requests. 
A web server returning '404' is going to close out connections much 
faster than the 'e-commerce/img/java-encrusted blicki' and is going 
to get more requests sent to it.  I'd think all the redirectors would 
do this as it provides automatic balancing of load across servers of 
differing performance.  Proper use of the application-layer health 
check should prevent this from being a real problem, but the 
performance overhead might be a bit much and I think there will still 
be cases where something in the blicki goes wacko outside of where 
the redirector looks.  External healthchecks could cover that 
(automated monitoring of the site that could go kick somebody to take 
a look maybe).
	Of course you can always just forget the application-layer health 
checks on the redirector and have something else take the server 
interface down when it thinks the server is behaving impolitely.


                -eric