VXLAN Encapsulation in Juniper Contrail

VXLAN is becoming de-facto encapsulation standard for overlay virtual networks (at least according to industry pundits and marketing gurus working for companies with VXLAN-based products) – even Juniper Contrail, which was traditionally a pure MPLS/VPN architecture uses it.

Not so fast – Contrail is using VXLAN packet format to carry MPLS labels between hypervisors and ToR switches.

Wait, What?

Juniper Contrail is still a pure MPLS/VPN solution, using L3VPN (RFC 4364) for routed traffic and EVPN for bridged traffic. Every packet forwarded between Contrail end nodes (hypervisors and/or PE-routers) has an MPLS label.

Contrail can use various encapsulation mechanisms to carry labeled customer traffic across the underlying IP transport network (they decided to use IP-based transport fabric instead of MPLS-based transport fabric):

  • MPLS-over-GRE – the traditional encapsulation mechanism supported by several router vendors, including Cisco and Juniper;
  • MPLS-over-UDP – a variation of MPLS-over-GRE that replaces GRE header with UDP header. A hash of the payload (entropy) is stored in the source UDP port to enhance ECMP load balancing across the underlying IP fabric;
  • MPLS-over-VXLAN, which uses VXLAN packet format but stores MPLS label in the VNI (Virtual Network Identifier) field.

In all three cases, the traffic within the current Contrail implementation never uses more than a single MPLS label (VPN label). The LDP or MPLS TE label is not needed due to end-to-end IP transport, and multi-label MPLS/VPN architectures (Carrier’s Carrier) aren’t supported.

Why would you need VXLAN?

By now you should be thoroughly confused and asking “why would anyone want to bastardize VXLAN to make it work like MPLS label stack?

The answer is very simple: imagine you have a hardware platform with third-party merchant silicon that supports VXLAN encapsulation but not MPLS-over-GRE encapsulation (or that supports VXLAN encapsulation way better than MPLS labels).

The only reasonable way forward is to use the encapsulation supported by hardware platform (VXLAN) and adjust the meaning of the VNI to whatever your control plane needs are. Does that make sense? It probably does to the engineers trying to squeeze out the last drops of performance from the chipset – but I wouldn’t want to be the one trying to troubleshoot the whole thing.

Looking for a bigger picture?

I briefly covered Contrail and Nuage VSP in the Following Packets across Overlay Virtual Networks part of the Overlay Virtual Networking webinar. For additional virtual networking and cloud networking architecture webinars, check out the cloud networking resources on ipSpace.net.


  1. Current version of Contrail uses VXLAN encapsulation to carry non-IP traffic between VMs across hypervisors. The VXLAN payload is the original Ethernet frame. An upcoming release will support use of VXLAN to carry IP and non-IP traffic between VMs and non-virtualized hosts connected to merchant silicon based TORs. The VXLAN payload will still be the original Ethernet frame.

    The EVPN + VXLAN implementation is based on the following:


    IOW, the VNI field in the VXLAN header is not used to store an MPLS label.

    On a related note, draft-sd-l2vpn-evpn-overlay does use the MPLS label field in BGP EVPN Routes to signal the VNI in the control plane. The data plane remains vanilla VXLAN.

    Disclosure: I work in the Contrail team at Juniper
    1. Straight from http://opencontrail.org/opencontrail-architecture-documentation/ (section

      "Second, The Virtual Network Identifier (VNI) in the VXLAN header is locally unique to the egress vRouter instead of being globally unique."

      Sounds like MPLS label to me. What you want to call that field is pure semantics ,)

      Or maybe your architecture document is out-of-date, in which case someone might want to fix it.
    2. The document refers to the implementation of locally significant VNIs as described in draft-sd-l2vpn-evpn-overlay. I agree that they are conceptually similar to MPLS labels.

      The point I would like to clarify is that per the above draft, VXLAN encap is used only when the destination prefix has a VNI associated with it e.g. a MAC address learnt via BGP EVPN route with Encapsulation Extended Community value VXLAN. It's not used if the destination prefix has an MPLS label associated with it e.g. IP prefix learnt via BGP L3VPN route.

      We've since discovered that merchant silicon TORs tend to only support globally unique VNIs. Hence newer releases of Contrail software will switch over to using globally unique VNIs. The section you referred to above will need to be updated at that point.
  2. Ivan, you sounds like network industry hipster sometimes. Keep it up! I like this ironic sense of humor.
Add comment