Imagine the Ruckus When the Hypervisor Vendors Wake Up

It seems that most networking vendors consider the Flat Earth architectures the new bonanza. Everyone is running to join the gold rush, from Cisco’s FabricPath and Brocade’s VCS to HP’s IRF and Juniper’s upcoming QFabric. As always, the standardization bodies are following the industry with a large buffet of standards to choose from: TRILL, 802.1ag (SPB), 802.1Qbg (EVB) and 802.1bh (Port extenders).

The only viable argument the whole industry has for the push toward large(r) layer-2 domains is VM mobility – if you want to migrate a live virtual machine across the data center and retain its sessions, it has to stay in the same VLAN due to simplistic virtual switches offered by hypervisor vendors (be it VMware, Microsoft, Citrix or open-source community) and lack of session layer in the TCP/IP stack.

Imagine one of the hypervisor vendors eventually wakes up, realizes a server has two external connections (storage and network), and delivers IP-over-IP or MAC-over-IP connectivity (MPLS, mGRE with NHRP or even LISP would be better due to easy integration with existing hardware devices, but most vSwitch vendors would probably want to avoid any control plane investment).

All of a sudden, all we need in the Data Center is layer-3 connectivity designed and implemented using the same mechanisms we’ve been using for the last 30 years to build the (somewhat scalable) Internet. The only reason for the layer-2 gold rush is gone.

My Wild Speculations

I’m guessing that most vendors have already seen the writing on the wall. Glacial progress of 802.1Qbg and the (so far) empty promises of EVB support we hear from some vendors would certainly indicate that.

TRILL seems to be completed, but nobody has implemented it yet (although I’m positive HP will follow through on its promises now that the standards are ratified). The only two jumping on the TRILL-ish bandwagon so far are Cisco and Brocade, each one with a proprietary implementation (FabricPath and VCS Fabric).

802.1aq seems to be in a slightly better shape, but then it addresses a number of Service Provider problems as well, so it’s worth implementing just to get rid of the Spanning Tree Protocol in your SP-focused gear.

In the end, it will be interesting to see how many of the promised flat-earth features will eventually get implemented, particularly by those vendors that don’t invest much in R&D.


  1. Xen and KVM do live migration over TCP. If your storage is located on an iSCSI volume for example, there is no need for a flat network. The L2 requirement for migration comes mostly from VMWare.
  2. Vincent, the "same VLAN" requirement comes from the VM NIC perspective - if you want to keep TCP sessions, the VM must keep the same IP address, which usually means its NIC must stay in the same VLAN.

    VMware can move VM as long as both ESX hosts see the same storage (regardless of how SAN connectivity works) and vMotion traffic runs over TCP ... but if you move a VM to another subnet, its Ethernet connectivity is gone.
  3. So what do you think is keeping them? Maybe the fact that they pitch mainly to server people (who don't really want to learn IP and especially (God forbid!) MPLS?
  4. They'll implement something sensible only after they realize the current kludges are limiting their growth. That's why Amazon went down this very route years ago.

    However, the commercial hypervisor vendors sell primarily to "small" environments (as compared to IaaS cloud providers like Amazon), where it doesn't really matter.
  5. Technology is a revolving door, where old becomes new, new becomes old, and around we go again when Vendors taste a flavour of the month coming along!
  6. Have you looked into Nicira at all? They are encapsulating the IP and I believe they have a kernel-level vSwitch module for Xen.

  7. I had a few interesting chats with Martin Casado (Nicira's CTO). When they're ready to tell me more about what they do, I'll blog about it.
  8. it just very well as listed....
  9. Looks like your players have been answered.
  10. Or, more accurately, "may" have been answered. :) These guys appear to be extremely secretive about what they do and how their stuff works.
  11. Pardon the dyslexia, pRayers that is, of course. ;)
  12. Citrix was the first to wake up. XenServer is enabled for OpenFlow already because it uses the Open vSwitch. Citrix is doing some interesting things with the OpenFlow startups (e.g. Nicira, BigSwitch).
  13. Force10 Networks has TRILL support now.
  14. Does it? Link to documentation?

    I know the support has been _announced_ around Interop, but that's not the same as shipping TRILL-compliant products.
Add comment