Leaf-and-Spine Fabric Myths (Part 3)
Evil CCIE concluded his long list of leaf-and-spine fabric myths (more in part 1 and part 2) with a layer-2 fabric myth:
Layer 2 Fabrics can't be extended beyond 2 Spine switches. I had a long argument with a $vendor guys on this. They don't even count SPB as Layer 2 fabric and so forth.
The root cause of this myth is the lack of understanding of what layer-2, layer-3, bridging and routing means. You might want to revisit a few of my very old blog posts before moving on: part 1, part 2, what is switching, layer-3 switches and routers.
Ready? Let’s go…
A bridged fabric relying on STP cannot have alternate paths between edge nodes (STP would block all but one of them). The usual trick to use is to make all spine switches appear as a single bridge to leaf switches using Link Aggregation (LAG) with LACP.
LACP was designed to be used between a pair of adjacent devices. To make it work across all spine switches you need multi-chassis LAG (MLAG). Vendors using independent control planes on spine switches usually don’t support more than two switches in an MLAG cluster, resulting in the two spines per fabric myth. Vendors using centralized control plane (= stackable switches) like HP IRF or Juniper Virtual Chassis don't have that limitation - a clear indication of how myopic some $vendor engineers can be.
Even more, some fabric solutions support LAG across more than two switches even when using independent control planes. The late Brocade was the first one I’m aware of; Juniper might be able to do something similar in EVPN deployments.
A routed fabric doesn’t have the same limitations, as it relies on a routing protocol to find alternate paths, and (if needed) on various extensions to traditional routing protocols to prevent flooding loops.
A routed fabric could work on layer-3 (IP) and use an overlay technology (for example, VXLAN) to transport layer-2 frames across the fabric, or on layer-2 (routing based on MAC addresses) using a standard (TRILL, SPB) or proprietary (FabricPath, VCS Fabric) technology.
I covered all of these options in the layer-2 fabrics section of the Leaf-and-Spine Fabric Architectures webinar and the fabric architectures part of the Data Center Fabric Architectures webinar Both webinars are part of Standard ipSpace.net subscriptionthe videos of the SPB part of that section is also available with Free ipSpace.net Subscription.
Once you get to the levels of bandwidth that warrant more than 2 spines, I would have thought you should be thinking long and hard about running a layer 2 fabric at all.
Any half-decent EVPN deployment should support that using ESI-LAG. Juniper certainly does.