RE: Thousands of hosts on a gigabit LAN, maybe not

[Sameer Khosla <skhosla () neutraldata com>]

You may want to look at CLOS / leaf/spine architecture.  This design tends to be optimized for east-west traffic, 
scales easily as bandwidth needs grow, and keeps thing simple, l2/l3 boundry on the ToR switch, L3 ECMP from leaf to 
spine.  Not a lot of complexity and scale fairly high on both leafs and spines.  

Sk.

-----Original Message-----
From: NANOG [mailto:nanog-bounces () nanog org] On Behalf Of John Levine
Sent: Friday, May 08, 2015 2:53 PM
To: nanog () nanog org
Subject: Thousands of hosts on a gigabit LAN, maybe not

Some people I know (yes really) are building a system that will have several thousand little computers in some racks.  
Each of the computers runs Linux and has a gigabit ethernet interface.  It occurs to me that it is unlikely that I can 
buy an ethernet switch with thousands of ports, and even if I could, would I want a Linux system to have 10,000 entries 
or more in its ARP table.

Most of the traffic will be from one node to another, with considerably less to the outside.  Physical distance 
shouldn't be a problem since everything's in the same room, maybe the same rack.

What's the rule of thumb for number of hosts per switch, cascaded switches vs. routers, and whatever else one needs to 
design a dense network like this?  TIA

R's,
John