nanog mailing list archives
Re: wow, lots of akamai
From: Charles Polisher <chas () chasmo org>
Date: Tue, 6 Apr 2021 09:16:38 -0700
On 4/5/21 10:23 PM, Robert Brockway wrote:
On Thu, 1 Apr 2021, Jean St-Laurent via NANOG wrote:What happened is that it would create a kind of internal DDoS and they would all timed out and give a weird error message. Something very useful like Error Code 0x8098808 Please call our support line at this phone number.If only there was a way to address the Thundering Herd problem before the cloud. :)This simple change to add 3 lines of code to add a random artificial boot penalty of few seconds, completely solve the problem.Bingo. Now, the trick is to catch this before it causes an self-DDoS.This is a problem that has been recognised for decades and this is unfortunately a good example of how operational experience is still not being distributed properly. Too many managers think that operational work is obvious and just a result of common sense. It isn't.
Same problem as disk drives powering up simultaneously in datacenters. SCSI drives have (had?) a random delay mechanism to distribute the initial power surge over a few seconds.
Current thread:
- RE: wow, lots of akamai, (continued)
- RE: wow, lots of akamai Jean St-Laurent via NANOG (Apr 01)
- Re: wow, lots of akamai Töma Gavrichenkov (Apr 01)
- Re: wow, lots of akamai Dave Brockman - DVS (Apr 02)
- Re: wow, lots of akamai Jared Mauch (Apr 02)
- Re: wow, lots of akamai Mike Hammett (Apr 02)
- Re: wow, lots of akamai Tom Beecher (Apr 01)
- RE: wow, lots of akamai aaron1 (Apr 01)
- Re: wow, lots of akamai Mark Tinka (Apr 02)
- RE: wow, lots of akamai aaron1 (Apr 02)
- Re: wow, lots of akamai Charles Polisher (Apr 06)