Continuing our bridging loops discussion Christoph Jaggi sent me another question:

Theoretically STP should avoid bridging loops, and yet you claim they cause data center meltdowns. What am I missing?

In theory, STP avoids bridging loops. In practice, there are numerous reasons STP got a bad name.

Blocked alternate paths. That’s a design choice you have to accept if you want to have plug-and-pray networking instead of proper routing protocols. Not much we can do here.

Forward-on-failure behavior. This is the only real grudge I have against STP as protocol – all links forward traffic until BPDUs cause some of them to be blocked.

If you insert a device that drops BPDUs in the forwarding path, or if a switch loses its control plane (for example, due to a memory leak), a forwarding loop is almost guaranteed.

A unidirectional link (due to bad transceiver or cable) could also result in a forwarding loop when the bridge that should have put the link in blocking state doesn’t receive the BPDUs (thanks to Antonio Ojea for pointing this out).

Slow link establishment. Vanilla STP waits 30 seconds before it starts forwarding traffic onto a link, and vendors ignorant of how networks work start sending their precious traffic as soon as they see layer-1 carrier on a server NIC, forcing networking vendors to implement all sorts of kludges like portfast and bpduguard.

The kludges implemented by networking vendors are not reliable. For example, BPDU Guard kicks in after the first BPDU is received, potentially resulting in a temporary forwarding loop before the first BPDU reaches the switch.

Too many kludges cause configuration errors. Understanding all possible kludges vendors implemented around STP and the relationship between them is hard, and even the big guys sometimes get it wrong.

Virtualization vendors drop BPDU frames. What could possibly go wrong if someone configures bridging between two VM NICs? It’s definitely better to pretend it’s someone else’s problem and blame the network instead of explaining to the sysadmin why his VM was kicked off the virtual switch after he made a stupid configuration error.

Some fabric vendors ignore(d) STP and propagate(d) BPDUs across the fabric, dramatically increasing the blast radius of any misconfiguration.

Want to Know More?

How Networks Really Work, Data Center Infrastructure for Networking Engineers and Leaf-and-Spine Fabric Architectures webinars discuss large-scale bridging and routing on layer-2, including SPB, TRILL, VXLAn and EVPN.