Re: Cloudflare is down

[danny () tcb net]

On 2013-03-04 08:09, Christopher Morrow wrote:
> On Mon, Mar 4, 2013 at 2:31 AM, Saku Ytti <saku () ytti fi> wrote:
> > I know lot of vendors are fuzzing with 'codenomicon' and they appear 
> > not to
> > have flowspec fuzzer.
>
> i suspect they fuzz where the money is ...
>
> number of users of bgp?
> number of users of flowspec?

While fuzzing of BGP[*] on the wire _may have identified some of this, 
there were many components involved (e.g., the DDoS attack on a 
customer's DNS servers that tickled their "attack profiler", their 
attack profiler was presumably confused about the suspect packet sizes 
as indicated in the presented "output signature", their operator didn't 
identify the issue before disseminating the recommended "signatures", 
JUNOS didn't barf when compiling the configuration (that'd be a big 
packet), a memory leak / thrashing triggered by the ingested flow_spec 
UPDATE crashed receiving routers, routers apparently recovered 
non-deterministically, etc..).

Leo's comments remind me of the The President's Commission to 
Investigate the Accident at Three Mile Island (TMI) findings, where 
pretty much everyone was blamed, but the operators were identified as 
ultimately culpable (in this case, presumably, _they also wrote the 
"attack profiler", although "they" may not have been precisely who 
deployed the policy).

For an interesting perspective of "normal accidents" derived from 
interactive complexity see [NormalAccidents], it's quite applicable to 
today's networks systems, methinks.

-danny

[NormalAccidents] Perrow, Charles, "Normal Accidents: Living with 
High-Risk Technologies", 1999.