EVPN-VXLAN Service Types

[Graham Johnston via NANOG <nanog () nanog org>]

Good day, NANOG.

I'm at the front end of an expected implementation of EVPN-VXLAN as the primary method to shift a network that is 
largely based on traditional Ethernet switching and spanning-tree to one that attempts to route traffic as often as 
possible, and where we want to separate the physical topology from the logical services. We are selecting EVPN-VXLAN as 
it seems to inherently provide for the Network Virtualization Overlay function, as well as routing since the entire 
underlay will be routed. As part of all the reading we are doing, and lab testing that is just about to commence, I'm 
trying to weigh the options around VLAN-based services and VLAN-aware bundle services. I know that the options aren't 
mutually exclusive, and that I can mix and match, at least I expect that this to be an option.

In case it matters, our implementation will initially involve VTEPs based on a mix of Juniper QFX5100, QFX5110, 
QFX5120, and EX4650 switches, as well as MX. Yes, I do recognize the RIOT capabilities that aren't present in the 
QFX5100.  From a basic FIB standpoint, we do believe that we are well below the quote limits in terms of hosts, routes, 
etc. I do believe that we've effectively weighed the use of VXLAN over MPLS. We currently believe that our use cases 
don't require some of the more advanced features and control knobs available in MPLS. We are also pragmatic and are 
trying to use the equipment that we have. We believe that the Trident ASICs in our devices are likely better suited for 
VXLAN than MPLS, despite the glossy datasheets quoting support for various MPLS features. Feel free to comment on this.

For internal use, I can see the VLAN-aware bundles as advantageous to group all our own services together in a single 
MAC-VRF, treat ourselves as a tenant. I'm not clear yet if I should be concerned or not about each switch that is 
involved in this EVI having to populate all entries into FIB. Our own use cases are likely of a small enough scale that 
it wouldn't matter in comparison to the positive outcomes. As for customer use cases, I can't yet see an advantage to 
VLAN-aware bundles as our customers don't interact with multiple VLANs where those individual VLANs are terminating on 
individual VTEP ports. The customer use cases feel more like a traditional Q-in-Q type activity that has us treating 
them as single outer VLAN, and thus the VLAN-based service seems more appropriate. I'm flat out ignoring the middle 
ground option of VLAN-bundle service as I can't see anything that seems compelling compared to the other two.

I know there is bunch that I don't know here. Am I focusing on the right two choices of the three service types? Do 
organizations regularly use both two that I am focusing on? How do you decide between the two models when provisioning 
an EVI? What gotchas await me with the Juniper equipment, or the Trident ASICs, that just aren't spelled out in the 
documentation? Answers to these questions and anything else you have to offer is appreciated.

Thanks in advance,
Graham