RE: Thousands of hosts on a gigabit LAN, maybe not

[c b <bz_siege_01 () hotmail com>]

If you need that kind of density, I recommend a Clos fabric. Arista, Juniper, Brocade, Big Switch BCF and Cisco all 
have solutions that would allow you to build a high-density leaf/spine. You can build the Cisco solution with NXOS or 
ACI, depending which models you choose. The prices on these solutions are all somewhat in the same ballpark based on 
list pricing I've seen... even Cisco (the Nexus 9k is surprisingly in the same range as branded whitebox). There is 
also Pluribus which offers a fabric, but their niche is having server procs on board the switches and it seems like 
your project involves physical rather than virtual servers. Still, the Pluribus could be used without taking advantage 
of the on board server compute I suppose.
I also recommend looking into a solution that supports VXLAN (or GENEVE, or whatever overlay works for your needs) 
simply because MAC is carried in Layer-3 so you won't have to deal with spanning tree or monstrous mac tables. But you 
don't need to do an overlay if you just segment with traditional VLANs.
I'm guessing you don't need HA (A/B uplinks utilizing LACP) for these servers?
Also, do you need line rate forwarding? Having 1,000 devices with 1Gb uplinks doesn't necessarily mean that full 
throughput is required... the clustering and the applications may be sporadic and bursty? I have seen load-testing 
clusters, hadoop and data warehousing pushing high volumes but the individual NICs in the clusters never actually hit 
capacity... If you need line-rate, then you need to do a deep dive with several of the vendors because there are 
significant differences in buffers on some models.
And... what support do you need? Just one spare on the shelf or full vendor support on every switch? That will impact 
which vendor you choose.
I'd like to hear more about this effort once you get it going. Which vendor you went with, how you tuned it, and why 
you selected who you did. Also, how it works.
LFoD
> Date: Sun, 10 May 2015 01:17:07 +0000
> From: johnl () iecc com
> To: nanog () nanog org
> Subject: Re: Thousands of hosts on a gigabit LAN, maybe not
>
> In article <CAHf3uWyPQn1NS_umjZ-zNuk3i5uFcZBu9L39b-crovG6yUm2qA () mail gmail com> you write:
> > Juniper OCX1100 have 72 ports in 1U.
>
> Yeah, too bad it costs $32,000.  Other than that it'd be perfect.
>
> R's,
> John