vMotion and VXLAN

A while ago I wrote “vMotion over VXLAN is stupid and unnecessary” in a comment to a blog post by Duncan Epping, assuming everyone knew the necessary background details. I was wrong (again).

Trying to understand my reasoning Jon sent me a very nice question:

vMotion is an exchange of data between hypervisor kernels, VXLAN is a VM networking solution. I get that, but in one of your videos you say VXLAN is a solution to meet the L2 adjacency requirements of vMotion.

If you design a L3 IP transport network ie. L3 from access to aggregation but you wanted to use vMotion then how could you do that unless you used an overlay technoology such as VXLAN to extend the vlan across the underlying IP network.

My somewhat imprecise claims often get me in trouble (this wouldn’t be the first time), let me try to straighten things out.

vMotion requires:

  • L2 adjacency between source and target hypervisor hosts for the port group in which the VM resides. Without the L2 adjacency you cannot move a live IP address and retain all sessions (solutions like Enterasys’ host routing are an alternative if you don’t mind longer traffic interruptions caused by routing protocol convergence time);
  • IP connectivity between vmkernel interface of hypervisor hosts (vMotion uses TCP to transport data between hypervisors). VMware always claimed that you need L2 connectivity between hypervisor hosts, and that vMotion between hosts residing in multiple subnets is unsupported (supposedly it became supported last year), but it always worked.

In other words, when you move a VM, its Ethernet NIC must reside in the same L2 segment after the move, but the vmkernel interfaces of the source and target hypervisor hosts can be in different subnets.

You can implement the VM NIC must be in the same L2 segment requirement with VLANs (which require end-to-end L2 connectivity) or VXLAN (which emulates L2 segments across L3 infrastructure).

Related webinars

Also, don’t forget to check out other virtualization webinars and the webinar subscription.

Recent posts in the same categories

8 comments:

  1. Anonymous 21 May 2014 12:49
    Also remember that distributed vSwitch is the boundary for vMotion. So even if you stretch L2 with VXLAN between two different vSwitches you will not be able to vMotion the VM. It is current vCenter Server limitation.
  2. Simon Hamilton-Wilkes 21 May 2014 14:30
    @fojta - Is that the case even if the port groups have the same name?
    Replies
  3. shah 21 May 2014 17:40
    How do we by pass the limitation of VMware which states that vmkernel interfaces of vMotion should be in same L2 segment. If we put this vmkernel interface behind VXLAN port-group it will be bit odd setup as you have traffic initiated from VMkernel gets encapsulated by another vmkernel interface.
    Replies
    1. Ivan Pepelnjak 21 May 2014 17:45
      You did read the whole blog post, did you?
  4. Arron 22 May 2014 03:23
    On the VMware L2 for VMotion topic, they seem to be moving a bit on that.

    There is an interesting piece by Doug Youd (all credit). "For ongoing [L3 Vmotion] support, it is recommended that users go through the RPQ process so VMware will validate the design. ” - VMware NSX Design Guide (page 14)"
  5. Anonymous 26 May 2014 01:13
    VMotion is just moving a bunch of data between hypervisor kernels. Does it matter if the connectivity is L2 or L3, though VMW best practices suggests to have a separate Infra-VLAN for VMotions.

    What is the problem running VMotion over L3 ?. Will it interfere with data traffic ?. One can always use QoS to handle such a scenario.
  6. Unknown 30 May 2014 12:27
    What happens during a vMotion over a VXLAN segment is explained in this clear blog post made by a senior Technical Marketing Manager at VMware, with very nice figures: blogs.vmware.com/vsphere/2013/07/vxlan-series-how-vmotion-impacts-the-forwarding-table-part-6.html
Add comment
Home
Sidebar