Re: VXLAN for WAN Pseudowires?

[Ignas Bagdonas <ibagdona.nog () gmail com>]

Hi there,

> However,
> this would mean a shift from VPLS to VXLAN.
On this particular sentence - it is incorrect to compare VPLS and VXLAN. 
One is a set of control plane mechanisms for discovery and tunnel setup 
and data plane encapsulations and multipoint reachability model, while 
the other is just a data plane encapsulation. VXLAN is equivalent to a 
(point to point) pseudowire used as a data plane component in VPLS.


> 2) Traffic engineering - we don't have a lot of requirement for this, but do
>     have a small number of customers who buy A and B circuits, and require them
>     to be routed across different paths on our network. This is easy with MPLS
>     using explicit LSPs, but we've not yet worked out how to achieve the same
>     thing in VXLAN.
VXLAN is a tunnel over a routed IP network. The path to reach the tunnel 
endpoint will define how VXLAN encapsulated payload is traversing over 
your network. If tunnel endpoint is reachable via explicitly routed LSP, 
the payload will follow that path. VXLAN by itself does nothing to 
influence the underlay path.

> So, my question to the community is - have any of you used VXLAN as a wide-area
> layer 2 transport technology?
Yes, there are such deployments. The number of such deployments is 
increasing.

> Any pros or cons? Gotchas? Scare stories?
> Recommendations? Am I trying to shoot myself in the foot?

VXLAN is a tunnelling mechanism. You seem to be looking for a end to end 
solution, for which VXLAN is a (rather small) component. You need to 
start from the requirements. What is the underlay technology being used? 
How much of traffic engineering functionality will be required, and how 
the particular underlay technology can implement it? What about ECMP 
support and requirements for its use - flow mapping to VXLAN tunnel is 
the same problem as flow mapping to any other tunnel in underlay ECMP 
environment. To build all of this is one thing, but to operate is quite 
the other - what about OAM requirements? VXLAN has no OAM support, and 
will not get one. If your operations systems rely on MPLS based OAM for 
service layer (eg, an MPLS based PW) - that will not be available. How 
the tunnels will be set up - is there a need to do discovery, or will 
all of it be orchestrated?

Two main limiting factors in VXLAN applicability are lack of payload 
type indicator - the tunnel can carry only a single type of payload 
(ethernet in the canonic definition, no possibility to use single tunnel 
for multiple user payload types without additional encapsulation), and 
no ability to indicate that the payload is not a user payload but some 
other special type like OAM (pseudowire control word would be an example 
of such indicator). This leaves VXLAN as a solution of limited 
applicability for anything else than original intended one - ethernet 
tunnel over IP underlay. For other uses VXLAN is of limited 
applicability, there may be (and in fact are) vendor specific extensions 
but there is no point in talking about interoperability then. This 
problem space is well understood and there are successors to VXLAN - 
Geneve is getting into good shape (there are vendor specific early 
implementations but it is too early to talk about interoperability at 
this time), plus different vendors have their own tunnel encapsulation 
mechanisms (VXLAN-GPE being a more visible one, however it is not going 
to be standardized soon).

The other interesting part to solve is the control plane. BGP based one 
will likely be the most practical option here. Combined with ability to 
tunnel multiple payload types this gives you a single control plane for 
L3VPN and point to point and multipoint L2VPN built on the same underlay 
with the single tunnelling mechanism. Your operations team will thank 
you violently for doing this.

Ignas