Category: Bridging

My latest spanning tree protocol (STP) posts generated numerous comments, some of them so relevant that I decided to summarize them into another blog post.

Weird Things Happen

The unidirectional link scenario mentioned by Antonio is pretty well known:

Continuing our bridging loops discussion Christoph Jaggi sent me another question:

Theoretically STP should avoid bridging loops, and yet you claim they cause data center meltdowns. What am I missing?

In theory, STP avoids bridging loops. In practice, there are numerous reasons STP got a bad name.

My friend Christoph Jaggi, the author of fantastic Metro Ethernet and Carrier Ethernet Encryptors documents, sent me this question when we were discussing the Data Center Fabrics Overview workshop I’ll run in Zurich in a few weeks:

When you are talking about large-scale VLAN-based fabrics I assume that you are pointing towards highly populated VLANs, such as VLANs containing 1000+ Ethernet addresses. Could you provide a tipping point between reasonably-sized VLANs and large-scale VLANs?

It's not the number of hosts in the VLAN but the span of a bridging domain (VLAN or otherwise).

Another week, another ExpertExpress session, as is often the case focusing on two data centers with stretched VLANs spanning both of them. However, this one was particularly irksome, as the customer ran a firewall cluster stretched across two locations.

I gave the customer engineers my usual recommendations:

Matjaž Straus started the SINOG 2 meeting I attended last week with a great story: during the RIPE70 meeting (just as I was flying home), Amsterdam Internet Exchange (AMS-IX) crashed.

Here’s how the AMS-IX failure impacted ATLAS probes (world-wide monitoring system run by RIPE) – no wonder, as RIPE uses AMS-IX for their connectivity.

One of the responses I got on my “What is Layer-2” post was

Ivan, are you saying to use L3 switches everywhere with /31 on the switch ports and the servers/workstation?

While that solution would work (and I know a few people who are using it with reasonable success), it’s nothing more than creative use of existing routing paradigms; we need something better.

My “What Is Layer-2 and Why Do You Need It?” blog post generated numerous replies, including this one:

Pretend you are a device receiving a stream of bits. After you receive some inter-frame spacing bits, whatever comes next is the 2nd layer; whether that is Ethernet, native IP, CLNS/CLNP, whatever.

Not exactly. IP (or CLNS or CLNP) is always a layer-3 protocol regardless of where in the frame it happens to be, and some layer-2 protocols have no header (apart from inter-frame spacing and start-of-frame indicator).

I’m constantly ranting against large layer-2 domains; recently going as far as saying “we don’t really need all that stuff.” Unfortunately, the IP+Ethernet mentality is so deeply ingrained in every networking engineer’s mind that we rarely ever stop to question its validity.

Let’s fix that and start with the fundamental question: What is Layer-2?

One of my readers is struggling with the aftermath of marketing gimmicks:

We will implement a new network soon, and we're discussing P-routers versus regular routers versus switches. I'm looking for arguments to go one way or the other.

TL&DR: there’s no difference between router and L3 switch.

Randall Greer left a comment on my Revisited: Layer-2 DCI over VXLAN post saying:

Could you please elaborate on how VXLAN is a better option than OTV? As far as I can see, OTV doesn't suffer from the traffic tromboning you get from VXLAN. Sure you have to stretch your VLANs, but you're protected from bridging failures going over your DCI. OTV is also able to have multiple edge devices per site, so there's no single failure domain. It's even integrated with LISP to mitigate any sub-optimal traffic flows.

Before going through the individual points, let’s focus on the big picture: the failure domains.