Re: Amazon network engineering contact? re: DDoS traffic

[John Weekes <jw () nuclearfallout net>]

Zach,

Yes, RTBH is used to distribute the null-routes that I mentioned.

Unfortunately, even brief saturation events lasting just 5-10 seconds (a 
typical amount of time to detect the loss, issue the null-route, and see 
the traffic start to fall off as it is distributed upstream) can cause 
real damage to those customers who are sensitive to latency and packet 
loss. So while null-routes limit the duration of the impact, they can't 
eliminate it entirely. And, of course, the actual target of the attack 
-- the now-null-routed IP address -- becomes unreachable, which was 
presumably the goal of the attacker.

-John

On 11/8/2018 12:54 PM, Zach Puls wrote:
> No idea about an Amazon abuse contact, but do you have RTBH communities enabled with your upstream provider(s)? As a 
> general practice, when you detect a (D)DoS attack in progress, it would help to automatically advertise that prefix to 
> your upstream(s) with the black-hole community. This would at least help mitigate the effects of the attacks when they 
> do occur, even if they come from a different source than AWS.
>
> Thanks,
>
> Zach Puls
> Network Engineer | MEF-CECP
> KsFiberNet
>
> -----Original Message-----
> From: NANOG <nanog-bounces () nanog org> On Behalf Of John Weekes
> Sent: Thursday, November 08, 2018 14:44
> To: nanog () nanog org
> Subject: Amazon network engineering contact? re: DDoS traffic
>
> We've been seeing significant attack activity from Amazon over the last two months, involving apparently compromised 
> instances that commonly send 1-10G of traffic per source and together generate Nx10G of total traffic. Even when our overall 
> upstream capacity exceeds an attack's overall size, the nature of load-balancing over multiple 10G upstream links means that 
> an individual link can be saturated by multiple large flows, forcing our systems to null-route the target to limit impact.
>
> We've sent an abuse notification about every traffic source to Amazon, and specific sources seem to stop their 
> involvement over time (suggesting that abuse teams are following up on them), but there is an endless parade of new 
> attackers, and each source participates in many damaging attacks before it is shut down.
>
> Is there anyone at Amazon who can help with an engineering solution in terms of programmatically detecting and 
> rate-limiting attack traffic sources, to our networks or overall? Or applying the kludge of a rate-limit for all Amazon 
> traffic to our networks? Or working with us on some other option?
>
> At least one other large cloud provider has an automatic rate-limiting system in place that is effective in reducing 
> the damage from repeat high-volume attacks.
>
> Emails to the Amazon NOC, peering contacts (since that would be another possible solution), and abuse department have 
> not connected me with anyone.
>
> Thanks,
> John