A Network Artist left a lengthy comment on my Brief History of VMware NSX blog post. He raised a number of interesting topics, so I decided to write my replies as a separate blog post.

Using Geneve is an interesting choice to be made and while the approach has it’s own Pros and Cons, I would like to stick to VXLAN if I were to recommend to someone for few good reasons.

The main reason I see for NSX-T using Geneve instead of VXLAN is the need for additional header fields to carry metadata around, and to implement Network Services Header (NSH) for east-west service insertion.

The simplest scenario where you need metadata is hierarchical BUM flooding (with per-subnet proxy) combined with dynamic MAC learning - you need to know who the original sender of the packet was, and you can’t get that from source IP address of the underlay packet because the packet was sent by the proxy.

Not sure about how many Server NICs can handle Geneve in HW and state of SR-IOV and DPDK in reference to Geneve.

I’m not sure about NIC support for Geneve, but DPDK definitely supports it - that’s what NSX-T uses in high-performance environments. Of course you have to use Intel NICs to make it work.

Also need to get my head around how Multicast will be handled in control and data plane.

There is no underlay multicast in NSX-T, probably because nobody ever bothered to implement it in Open vSwitch. BUM flooding is done either from the originating hypervisor or through per-subnet proxies.

As of NSX-T 2.5 there is no support for overlay multicast, at least across subnet boundaries - you can send whatever you wish (AppleTalk anyone?) within a logical switch.

How would you correlate underlay vs. overlay stats for visibility, performance mgmt. & troubleshooting ?

We’ve been facing the same problem from the days we started connecting routers to Frame Relay instead of leased lines to build WAN networks. We had the same problems with IPsec tunnels, GRE tunnels, MPLS-over-GRE-over-IPsec-over-MPLS tunnels… Some things never change, and while we can’t expect miracles at least there’s vast prior experience that might help you when reading the pretty good NSX-T Data Center Troubleshooting Guide.

On a totally unrelated topic, I can’t fathom why people keep raising the same objections every time they’re facing an age-old situation. It’s not like complaining would ever make something go away. Learn how to deal with it, and get on with your life.

Want to know more about NSX-T? I’m starting deep dive into NSX-T in mid-November 2019, and all you need to join the live sessions is a Standard ipSpace.net Subscription.