One of my readers sent me a question along these lines…

VXLAN Network Identifier is 24 bit long, giving 16 us million separate segments. However, we have to map VNI into VLANs on most switches. How can we scale up to 16 million segments when we have run out of VLAN IDs? Can we create a separate VTEP on the same switch?

VXLAN is just an encapsulation format and does not imply any particular switch architecture. What really matters in this particular case is the implementation of the MAC forwarding table in switching ASIC.

Ignoring for the moment that we have to map VXLAN segments into physical segments (= VLANs) at some point in time, each VXLAN segment is a separate bridging (L2 forwarding) domain, which means that it needs a separate L2 forwarding table.

Multiple forwarding tables could be implemented with multiple data structures - a perfect solution for a software-based forwarding. There might be some scalability snag hiding deep within Linux network stack implementation (like realistic limits on the number of devices), but it SHOULD be possible to create a gazillion Linux bridges and place each one of them in a different VXLAN segment.

It’s hard to use the same trick in hardware. We could create separate data structures, but they would still sit in the same TCAM, so we need an extra field to differentiate them - instead of looking up a MAC address, the hardware lookup operation would search for a combination of (Table-ID,MAC). The number of bridging domains supported by switching ASIC is thus limited by the size of Table-ID part of the lookup table… and for historic reasons that happens to be 12 bits.

Long story short: No matter how many bits we assign to segment ID in the packet header, we’re still limited by how many bits hardware manufacturers allocate to bridging table ID in their ASIC lookup tables. Creating another VTEP won’t help - in the end all bridged packets use the same MAC lookup table.