Category: Bridging

A networking engineer (let’s call him Joe¹) sent me an interesting challenge: they built a data center network with Cisco switches, and the switches flood LLDP packets between servers.

That would be interesting by itself (the whole network would appear as a single hub), but they’re also using DCBX (which is riding in LLDP TLVs), and the DCBX parameters are negotiated between servers (not between servers and adjacent switches), sometimes resulting in NIC resets².

This blog post describes another “OMG, this cannot possibly be true” scenario discovered during the netlab VRRP integration testing.

I wanted to test whether we got the nasty nuances of VRRPv3 IPv6 configuration right on all supported platforms and created a simple lab topology in which the device-under-test and an Arista cEOS container would be connected to two IPv6 networks (Arista EOS is a lovely device to use when testing a VRRP cluster because it produces JSON-formatted show vrrp printouts).

Most platforms worked as expected, but Aruba CX, Cumulus Linux with NVUE, and Dell OS10 consistently failed the tests. We were stumped until Jeroen van Bemmel discovered that the Arista container forwards IPv6 router advertisements between the two LAN segments.

It all started with a netlab issue describing different interpretations of VLAN 1 in a trunk. While Cumulus NVUE (the way the netlab configuration template configures it) assumes that the VLAN 1 in a trunk is tagged, Arista EOS assumes it’s the native VLAN.

At that point, I should have said, “that’s crazy, we shouldn’t allow that” and enforce the “VLAN 1 has to be used as a native VLAN” rule. Alas, 20/20 hindsight never helped anyone.

TL&DR: Do not use VLAN 1 in VLAN trunks; if you have to, use it as a native VLAN.

Béla Várkonyi wrote a succinct comment explaining why so many customers prefer layer-2 VPNs over layer-3 VPNs:

The reason of L2VPN is becoming more popular by service providers and customers is about provisioning complexity.

Dan left an interesting comment on one of my previous blog posts:

It strikes me that the entire industry lost out when we didn’t do SPB or TRILL. Specifically, I like how Avaya did SPB.

Oh, we did TRILL. Three vendors did it in different proprietary ways, but I’m digressing.

Johannes Resch submitted the following comment to the Is Dynamic MAC Learning Better Than EVPN? blog post:

I’ve also recently noticed some vendors claiming that dataplane MAC learning is so much better because it reduces the number of BGP updates in large scale SP EVPN deployments. Apparently, some of them are working on IETF drafts to bring dataplane MAC learning “back” to EVPN. Not sure if this is really a relevant point - we know that BGP scales nicely, and its relatively easy to deploy virtualized RR with sufficient VPU resources.

While he’s absolutely correct that BGP scales nicely, the questions to ask is “what is the optimal way to deliver a Carrier Ethernet service?”

Justus sent me an email with an interesting link:

Since you love to make comparisons to the good ol’ thick yellow cable while I as a mid-30 year old adult have no idea what you are talking about: Computerphile made a video about Ethernet on the occasion of its 50th birthday. The university of Nottingham got the chance to show their museum pieces :-) (about 8:45 min).

Thanks a million!

The Dynamic MAC Learning versus EVPN blog post triggered tons of interesting responses describing edge cases and vendor bugs implementation details, including an age-old case of silent hosts described by Nitzan:

Few years ago in EVPN network, I saw drops on the multicast queue (ingress replication goes to that queue). After analyzing it we found that the root cause is vMotion (the hosts in that VLAN are silent) which starts at a very high rate before the source leaf learns the destination MAC.

It turns out that the behavior they experienced was caused by a particularly slow EVPN implementation, so it’s not exactly the case of silent hosts, but let’s dig deeper into what could happen when you do have silent hosts attached to an EVPN fabric.

I got this question from one of my readers:

Why are OSPF and BGP are more complex than STP from a designer or administrator point of view? I tried everything to come to a conclusion but I couldn’t find a concluded answer, ChatGPT gave a circular loop answer.

There are numerous reasons why a protocol, a technology or a solution might be more complex than another seemingly similar one (or as Russ White would have said, “if you haven’t found the tradeoffs, you haven’t looked hard enough”):

One of my readers worried about the control-plane-induced MAC learning lag in EVPN-based networks:

In all discussions about the advantages/disadvantages of VXLAN/EVPN, I can’t find any regarding the lag in learning new macs when you use the control plane for mac learning.

EVPN is definitely slower than data plane-based dynamic MAC learning (regardless of whether it’s done in hardware or software), but so is MLAG.