Building Network Automation Solutions
6 week online course starting in September 2017

Building a L2 Fabric on top of VXLAN: Arista or Cisco?

One of my readers working as an enterprise data center architect sent me this question:

I've just finished a one-week POC with Arista. For fabric provisioning and automation, we were introduced to CloudVision. My impression is that there are still a lot of manual processes when using CloudVision.

Arista initially focused on DIY people and those people loved the tools Arista EOS gave them: Linux on the box, programmability, APIs… However, if you want to enter the traditional enterprise segments where people don’t want to build their own tools or play a system integrator, there’s an enormous amount of work to be done to implement everything enterprise users might want to use and package it in a form factor that looks great in PowerPoint.

Unfortunately, most network management/provisioning solutions built by hardware vendors focus more on what sells to CxO (and looks great in PPT) than on the needs of ops people. Some people claim that Cisco’s Nexus Fabric Manager might be better than that – if you have hands-on experience, please write a comment.

However, that was not the only challenge my reader encountered. He said…

Also, the so-called VXLAN control plane from Arista with CVX is certainly inferior to Cisco's BGP EVPN VXLAN control plane.

Update 2016-06-01: Rewrote the following paragraph which incorrectly stated that CVX doesn't provide control-plane MAC learning

BGP EVPN and Arista's CVX provide control-plane MAC address learning, EVPN using a standard protocol that's potentially interoperable across vendors, CVX with an Arista-specific protocol. However, at least Cisco's implementation of EVPN provides end-to-end routing based on host routes (which also enables intra-subnet proxy ARP functionality) and symmetrical IRB (which could limit the flooding scope), and I found nothing equivalent in currently-shipping EOS.

However, keep in mind that the biggest problem you have when building a layer-2 fabric on top of an VXLAN overlay is provisioning:

  • mapping VLANs into VXLAN VNIs;
  • configuring subnets;
  • configuring anycast gateways.

You have to do these tasks “manually” (or using an automation/orchestration solution) on either Arista or Cisco (or use the APIC controller with Nexus 9000 fabrics).

The real benefits of EVPN as compared to flood-and-learn approach to VXLAN are automatic discovery of VTEPs and replacement of flood-and-learn MAC address discovery with deterministic BGP-based propagation of information.

Need more information?

  • Lukas Krattiger discussed the details of EVPN with VXLAN in Leaf-and-Spine Fabric Architectures webinar, which also covers numerous ways of building layer-2 and layer-3 data center fabrics;
  • Data Center Fabrics webinar covers VXLAN implementations from Arista, Brocade, Cisco, Dell, HP and Juniper, and it’s updated every year to keep the information up-to-date;
  • For a deep dive into data center design challenges, join the Building Next-Generation Data Center online course.

13 comments:

  1. If you choose Cisco ACI, you don't choose the overlay i.e. evpn... Not sure about Arista. This is one of my concerns with vendors all in one solutions is you don't choose underlying protocols for encapsulation, routing, etc...It is just supposed to work, leaving you with primarily only the controller to interact with. I still think there is much to be said about rolling your own vs. just trusting that a vendor did all the hard work for you and the customer gets a pristine, always on "black box" which ops manages as a network fabric i.e. clos

    ReplyDelete
    Replies
    1. I might be wrong, but it's my understanding that in Arista's case you see what's going on (at least you can look at the resulting switch configuration).

      Delete
  2. Use OpenStack on top of either one of these solutions to get away from many of the "manual" headaches of VLAN/VXLAN mapping.

    ReplyDelete
    Replies
    1. And get totally different headaches instead.

      Delete
  3. it is not true for "cvx has no control plane", it can adv and withdraw mac to vtep binding, just like bgp. Only problem for cvx is it is proprietary.

    ReplyDelete
    Replies
    1. I know that works with OpenStack or NSX controller, but does the same functionality exist in the standalone CVX deployment?

      Delete
    2. This seems to confirm that it does work without openstack or nsx:
      https://eos.arista.com/vxlan-without-controller-for-network-virtualization-with-arista-physical-vteps/#6_VXLAN_implementation_differences

      Delete
    3. Hi Ivan,

      With Standalone CVX - it does adv and withdraw MAC-vtep binding. With plugins like Openstack & NSX on cvx helps one to deploy with different orchestration.

      Delete
    4. Updated the paragraph. Thank you all for pointing this out!

      Delete
  4. Provisioning goes hand in hand with inventory, having a good and flexible inventory model here would be the key. So :) actually neither cisco nor arista could convince me that they are good in provisioning service fulfillment kind of business.

    ReplyDelete
  5. Regarding ACI.

    You are able to see what ever you want regarding VXlan and the mac/topo information. You have a normal NX-OS cli for debugging and there is no magic other then proven protocols melted together as a packaged solution.
    What you can´t see is show running-config :) of the fabric it self.
    ACI will soon be able to peer EVPN with the outside!!

    ReplyDelete
  6. Joachim Jerberg Jensen11 June, 2016 22:11

    Ivan.. Pls look at Cisco VTS controller.

    It does all the overlay provisioning for you, with integration to vCenter or OpenStack. It even comes with a software forwarder for doing the VTEP all the way on the host, called VTF (based on XR code and VPP dataplane), which increases scale om leaf VNI mappings, as well as increases flexibility.

    VTS also acts as EVPN route-reflector for VTEP's to overcome scaling issues on the switches RIB and controlplane in very large deployments. Further it can provision the DCI routers (if ASR9K)

    Joachim Jerberg Jensen
    Cisco Systems Engineer

    ReplyDelete
  7. Hi Ivan

    I found the Avaya SPBM solution "right on the money" to that L2+ fabric
    Will do deploy Avaya SPBM?

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.