Re: Web caching liability

[Dean Gaudet <dgaudet-list-nanog () arctic org>]

Nope this post can't be configured into your router Randy. 

On Fri, 2 Jan 1998, Richard Welty wrote:

> Brian Horvitz wrote:
>
> > Has anyone else exerienced problems with AOL's web proxy servers?  We
> > switched the IP address of a web site about 5 months ago, and we are still
> > getting about 10,000 hits a day from AOL on the old address.  We dropped a
> > note to them a ways back but didn't get much of a response.
>
> they've been problems for a long time; some of their proxy servers 
> hold cache data for far too long. i have some web discussion pages 
> that are effectively useless for aol subscribers because aol even 
> caches cgi generated stuff with obvious cgi related extensions like 
> ".cgi" and ".pl".

I'm taking a wild guess that your CGI responses include neither
Last-Modified or Expires headers.  HTTP/1.0 (rfc1945) doesn't define what
a proxy cache is to do in this situation, so AOL is compliant.  HTTP/1.1
(rfc2068) section 13.2.4 gives more details on this, but that's not what
AOL implements.

Not to say that AOL's caches aren't littered with other standards
violations. 

The DNS thing isn't something that only AOL is innocent of.  Netscape
navigator up through the 3.x versions (I haven't tested the 4.x versions,
they may have fixed it) caches DNS responses for the lifetime of the
browser.  Given that some folks on stable unix machines are able to keep
their browser open for months this sucks.  One might argue it has
something to do with the lack of timeout information in the
gethostbyname(3) API.  I've always taken the opinion that DNS changes for
webservers regardless of your ttl values, will take at least two weeks and
you better plan to either lose some of the hits or run extra IP aliases
with a tunnel from the old addresses or whatnot. 

Dean