Sunset of VXLAN. Really?

A lightning talk at the recent RIPE77 conference focused on shortcomings of VXLAN and rise of Geneve.

So what are those perceived shortcomings?

No protocol identifier – a single VXLAN VNI cannot carry more than one payload type. Let me point out that MPLS has the same shortcoming, as does IPsec.

Original VXLAN was envisioned for carrying Ethernet payload only. Correct. There are other whatever-over-IP encapsulation techniques for other use cases, many of them with widespread deployment.

No inband OAM mechanism. Do we really need a new one? What’s wrong with sending CFM frames between VTEP MAC addresses… varying TTL values if want to track underlay path?

No extensibility. VXLAN has plenty of reserved fields in its header, but no extensibility… but do we really want another IPv6 headers fiasco? Add enough complexity and extensibility to a tunneling protocol and you’ll have another Fernando Gont running circles around you in no time.

Don’t get the reference? Search for IETF drafts and RFCs Fernando wrote and watch his presentations (I attended several of them at Troopers and Slovenian IPv6 Summit).

I don’t understand the obsession with universally extensible tools. There’s a reason every craftsman has a toolbox of tools that are just right for the job instead of a giant Swiss Army Knife.

However, don’t worry – IETF NVO3 working group has an answer: Geneve. It’s hardware-friendly (so the lightning talk) and has extensible header that can be up to 260 bytes long. I’ve heard that before… and it didn’t end well – we got to a point where software developers gave up on using smart NIC features because of all the underlying complexity.

I can’t help but being reminded of RFC1925 Rule 5… and even if I’m wrong (and I would love to be), do remember that every time someone designs a universal extensible protocol, you’ll be paying for the increased complexity and associated security SNAFUs of every device using it.

8 comments:

  1. Is "Geneve" an acronym for "Yet another network protocol"?
    Replies
    1. It's worse - it's an acronym for "yet another data plane encapsulation because the N existing ones aren't good enough". Should have included this link somewhere in the text:

      https://xkcd.com/927/
    2. I already knew about the cartoon :D So now I got the point ;)
  2. It would be nice to have VXLAN work across multiple vendors.
  3. Heya Ivan, great post as always.

    Got kind of a dumb question in regards to the first point that you raised (No protocol identifier). In MPLS, there seems to be new developments (one in draft named draft-xu-mpls-payload-protocol-identifier-05, and one seemingly in RFC7274 but this one is not exactly what is being talked about here) that do indeed offer that type of functionality. Albeit I will agree that it's likely pretty much not going to be used by vendors all that often. That all being said though, doesn't tacking more onto the MPLS label kinda defeat the purpose of it being the low overhead forwarding that it was meant to be in the first place? To me I see VXLAN kinda doing the same thing (over IP though) with Ethernet specific payloads. I don't see how it truly is a shortcoming to be honest, especially since one can signal a protocol payload type using LDP and RSVP.

    Just some thoughts I had. Here's hoping it's not detracting from your post here.
    Replies
    1. Totally agree with you - the payload type shouldn't matter... unless you want P routers to act on payload within MPLS frames. Common examples: ECMP load balancing based on L3/4 headers, or ICMP replies on TTL expiration.

      Since there's no TTL in Ethernet networks, and VXLAN uses UDP to capture payload entropy, these two common examples don't apply to VXLAN at all.
  4. I'll say it, time and time again, standards are generally an inch deep, and a mile wide, covering the most "common" of use cases. It has been networking vendors who have traditionally extended upon these standards to solve more complex, and niche, problems, or sometimes just providing something new that may solve other problems. Either way, why are people so concerned with the data-plane encapsulation? I think most of us can agree it is the control-plane where the real "magic" happens and where we start to develop, and deploy, real solutions for customers.

    I have a feeling there is one specific company out there who has a vested interest in seeing Geneve "take off", just one in particular. Geneve offers the promise of "interoperability" because it'll be "standard" and "flexible". Well, people said the same of the many spanning-tree protocols and I've, more than I can count, found two vendors couldn't even run MST in harmony. I don't see where VXLAN, or MP-BGP EVPN too, fell short to be "interoperable" among vendors, if anything, it is a vendor "thing" to work out, not the data-plane encap protocol. Just my 2 cents...
  5. Well there’s one huge problem of VXLAN though that was not mentioned in the presentation.

    And that is the inability to stack VXLAN headers
    -This is why MPLS is so successful -you don’t need to have a field in your tunnel header for every possible future feature -just need to be able to stack simple headers on top of each other.
    As a matter of fact that’s how VPNs (a.k.a micro-segmentation) or Traffic-Engineering (a.k.a service-chaining) works in MPLS.

    In VXLAN in order to achieve micro-segmentation all one is left with are proprietary hacks to the VXLAN header or lengthy access-lists.
    In MPLS if we don’t want A to talk to B we just don’t allow them to learn about each other’s MACs/IPs (put them in separate VRFs) -saves resources, we abandoned the model of everybody in common table and maintaining ACLs all over the place like good 15 years ago.

    Same goes for service-chaining,
    In SR we analyse the packet once on ingress and then use stack of headers to define the service chain
    In VXLAN all one is left with is again lengthy access-list and policy-based routing at every single hop in the chain.

    Oh I forgot resources at the edge are “virtual” = “endless” so why should I care about the length of the ALCs or size of the MAC tables right? …
    Maybe I’m old fashioned but I just happened to like elegant solutions in favor of the brute-force ones.

    Anyways I don’t get these DC folks reinventing the wheel all over again and again even though SP folks have had solutions for all these problems for well over a decade.

    adam

Add comment
Sidebar