Interfacing Overlay Virtual Networks with MPLS/VPN WAN

During my ExpertExpress engagements with engineers building multi-tenant cloud infrastructure I often get questions along the lines of “How do I integrate my public IaaS cloud with my MPLS/VPN WAN?” Here are a few ideas.

Don’t Overcomplicate

Let’s eliminate the trivial options first.

  • If your our public cloud offers hosting of individual VMs with no per-customer virtual segments, use one of the mechanisms I described in the Does It Make Sense to Build New Clouds with Overlay Networks? post and ask the customers to establish a VPN from their VM to their home network.
  • If your public cloud offers virtual private networks, but you don’t plan to integrate the cloud infrastructure with a multi-tenant transport network (using, for example, MPLS/VPN as the WAN transport technology), establish VPN tunnels between the virtual network edge appliance (example: vShield Edge) and customer’s VPN concentrator.

The rest of this post applies to multi-tenant cloud providers that offer private virtual networks to their customers and want to integrate those private networks directly with the MPLS/VPN service they offer to the same customers.

VLAN-based virtual networks

Many public cloud deployments use the “legacy” VLAN-based virtual network approach. Interfacing these networks with MPLS/VPN is trivial – create VLAN (sub)interface in a customer VRF for each outside customer VLAN on data center WAN edge PE-routers (Inter-AS Option A comes to mind).


Simple VLAN-based connectivity

Overlay virtual networks without MPLS/VPN support

If you use overlay virtual networking technology that has no integrated MPLS/VPN support (example: Cisco Nexus 1000V, VMware vCNS, VMware NSX, Hyper-V, OpenStack Neutron OVS plugin with GRE tunnels), you have to use VLANs as the demarcation point:

  • Create a VLAN per customer;
  • Use a VM-based appliance (firewall, load balancer) or L2/L3 gateway to connect the customer’s outside overlay virtual network with the per-customer VLAN;
  • Read the previous section.

Connecting overlay virtual networks and MPLS/VPN WAN through L2 gateway

Direct integration with MPLS/VPN infrastructure

Some overlay virtual networking solutions (Juniper Contrail, Nuage Virtualized Services Platform) communicate directly with PE-routers, exchanging VPNv4 routes via MP-BGP and using MPLS-over-GRE encapsulation to pass IP traffic between hypervisor hosts and PE-routers.

Integrating these solutions with the MPLS/VPN backbone is a trivial undertaking – establish MP-BGP sessions between the overlay virtual network controllers and WAN edge PE-routers. I would use Inter-AS Option B to establish a demarcation point between the cloud infrastructure and WAN network and perform route summarization on the PE-router (it doesn’t make much sense to leak host routes created by Contrail solution into the WAN network).


MPLS/VPN integration with Juniper Contrail

VM-level integration

If you don’t want to use one of the MPLS/VPN-based overlay virtual networking solutions (they both require Linux-based hypervisors and provide off-the-shelf integration with OpenStack and CloudStack), use a VM-based PE-routers. You could deploy Cisco’s Cloud Services Router (CSR) as a PE-router, connect one of its interfaces to a VLAN-based network and all other interfaces to customer overlay virtual networks.


Using Cisco CSR as MPLS/VPN PE router

The number of customer interfaces (each in a separate VRF) on the CSR router is limited by the hypervisor, not by CSR (VMware maximum: 10).

More Information

Visit SDN, Cloud or Virtualization resources on ipSpace.net, or get in touch if you need design or deployment advice.

Individual webinars you might find useful include:

9 comments:

  1. Would there be any firewalls where the Cloud infra. and MPLS/VPN network connect to each other?
    Replies
    1. Depends on the service definition.
  2. Besides Linux-based hyper-visors like KVM and Xen, Nuage VSP also supports VMWare ESXi as hyper-visor, and integrates with vCenter. This is what UPMC is using as was announced yesterday: http://www.nuagenetworks.net/press-releases/upmc-selects-nuage-networks/
    Replies
    1. Hi Jeroen,

      Thanks for the comment. It's nice to hear that you're working on ESXi support (btw, you might want to mention that you work for Nuage), but the press release you quote doesn't mention vSphere, VMware or ESX, and is thus totally irrelevant to the technical audience of this blog ;)

      Finally, it seems your products are moving in the right direction, and I would definitely consider writing about them ... as soon as I get my hands on shipping product documentation (hint, hint).

      Kind regards,
      Ivan
    2. Hi Ivan,
      Indeed I work for Nuage, something I take for granted - but you are right, I could have mentioned it explicitly. Unfortunately there are few public sources on Nuage for a technical audience like yours, I'm hoping we can change that soon. In the mean time your comment made me realize that perhaps we aren't as clear on what we are doing as we should be. The link was intended as some form of proof that it's not just me saying this; perhaps I should have used https://www.youtube.com/watch?v=S9iDUm07_Zo which shows Nuage VSP integration with VMWare about 20 minutes into the demo.
  3. Hi Ivan,

    Thanks for listing out all the options that are available.

    Is it not that "Direct integration with MPLS/VPN infrastructure" seems a better approach for Inter-DC & Hybrid cloud cases ?. This solution seems like a HW based, removes L2 scaliability, VLAN scalability & VLAN provising issue ?

    -Bhargav
  4. Nuage solution also supports direct integration with MPLS/VPN using VXLAN encapsulation to pass IP traffic between hypervisor hosts and PE-routers. VXLAN can be terminated directly into VPLS or IPVPN service on ALU SR7750 router. EVPN used as a control plane in this case. XMPP for service provisioning
  5. can we do Direct integration with cisco ACI ?
    Replies
    1. You can definitely do EVPN to MPLS/VPN integration (on Nexus 7x00 IIRC), not sure about ACI-to-MPLS/VPN.
Add comment
Sidebar