RE: 10G CPE w/VXLAN - vendors?

[Adam Thompson <athompson () merlin mb ca>]

The redundant links to the customer site that traverse independent underlay carriers, and in some cases, equal-cost 
paths that we want to load-balance across, are the hard part.  I’m not going to trust STP for that, and we aim for 
<3sec failover where we do have redundant paths.  ERPS can handle the failover, but not the load-balancing.  Any 
L2-over-L3 encapsulation protocol can handle the failover + ECMP features, but I need to do it at ~10G (~20G if ECMP) 
wire speed.

We provide IaaS services to our customers, which is why we’re stretching VLANs to them in the first place.  Viewed from 
the IaaS perspective, this is a bunch of DC-DC connections… but relative to the overall network, the customer-prem 
devices fall into the traditional “CPE” category.  (Most customers either just plug in bare fiber, or they connect to 
an intermediate carrier’s CPE.)

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN logo]]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01D99ECB.C1CEDE50]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athompson () 
merlin mb ca>

From: Joe Freeman <joe () netbyjoe com>
Sent: Wednesday, June 14, 2023 2:16 PM
To: Adam Thompson <athompson () merlin mb ca>; nanog <nanog () nanog org>
Subject: Re: 10G CPE w/VXLAN - vendors?

I think you’re probably overthinking this a bit.

Why do you need to extend your vxlan/evpn to the customer premise? There are a number of 1G/10G even 100G CPE demarc 
devices out there that push/pop tags, even q-in-q, or 802.1ad. Assuming you have some type of aggregation node you 
bring these back to, tie those tags to the appropriate EVPN instance at the aggregation point. Don’t extend anything 
but a management tag and an S-tag essentially to the device at the customer premise.

You can even put that management tagged vlan in it’s own L3 segment, or a larger L3 network and impose security. This 
way you’re not exposing your whole service infrastructure to a bad actor that might unplug your cpe device and plug 
into your network directly.



From: NANOG <nanog-bounces+joe=netbyjoe.com () nanog org<mailto:nanog-bounces+joe=netbyjoe.com () nanog org>> on behalf 
of Adam Thompson <athompson () merlin mb ca<mailto:athompson () merlin mb ca>>
Date: Wednesday, June 14, 2023 at 2:52 PM
To: nanog <nanog () nanog org<mailto:nanog () nanog org>>
Subject: 10G CPE w/VXLAN - vendors?
Hello, all.
I’m having difficulty finding vendors, never mind products, that fit my need.

We have a small but growing number of L2 (bridged) customers that have diverse fiber paths available, and, naturally, 
want to make use of them.
We have a solution for this: we extend the edge of our EVPN VXLAN fabric right to the customer premise.  The 
customer-prem device needs 4x10G SFP+ cages (2 redundant paths, plus LAG to customer), and the switches we currently 
use, Arista 7020Rs, are quite expensive if I’m deploying one one per customer.  (Nice switches, but overkill here – I 
don’t need 40/100G, and I don’t need 24 SFP+ ports.  And they still take forever to ship.)

We use RFC7438 §6.3 “vlan-aware-bundle” mode, not §6.1 “vlan-based” mode, which limits our choices somewhat.  I might 
be willing to entertain spinning up a separate VXLAN mesh using RFC7438 §6.1 (“vlan-based”) and static VTEPs if it 
saves me a lot of pain.

However, I’m having trouble finding small & cheaper 1U (or even desktop/wallmount) devices that have 4 SFP+ cages, and 
can do VXLAN, in the first place.
Who even makes CPE gear with SFP+ ports?  (Other than Mikrotik CRS309-1G-8S+IN / CRS317-1G-16S+RM, which are nice, but 
our policy requires vendor support contracts, so… no-go.)

Vendors?  Model#s, if you happen to know any?

Reply here or privately, whatever floats your boat – any pointers appreciated!

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN logo]]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01D99EC2.B891B0A0]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athompson () 
merlin mb ca>