We’ve seen several interesting data center fabric solutions during the Networking Tech Field Day presentations, every time hearing how the new fabric technologies (actually, the shortest path bridging part of those technologies) allow us to shed the yoke of the Spanning Tree monster (see Understanding Switch Fabrics by Brandon Carroll for more details). Not surprisingly we wanted to know more and asked the obvious question: “and how would you connect the switches within the fabric?”
The vendors were quick to assure us that “we can use any topology we want.” We also heard the buzzwords like hypercube, Clos, daisy chain and ring, and the promises like “you just plug it in ... it just works!” What they usually forgot to mention was the fact that removing the rigid requirements of spanning tree protocol doesn’t magically alleviate the need for proper network design.
Brandon has gracefully allowed me to use a picture from his blog post to illustrate the problem. Imagine you build a network shown in the following diagram. Because you’re using a fabric technology (be it TRILL, SPB, FabricPath or something else), no ports are blocked and you should be able to use all the bandwidth in the network ... but that simply won’t happen.
You see, the shortest path bridging technologies behave almost exactly like routing, and (like their name indicates) they give you shortest path bridging. All the traffic between A and E will still go over the B-C link because that’s the shortest path. The path A-B-D-C-E is longer and won’t be used.
The B-D and C-D links would be used if there would be other devices attached to D, but I hope you get my point – shortest path bridging technologies are no better than routing.
Just because the shortest path bridging technologies provide routing-like behavior at MAC layer doesn’t mean that you can wire your network haphazardly. Fortunately, you can fall back to the age-old rules of properly designed routed networks ... and guess what: they usually prescribe a hierarchical structure with edge, aggregation and core. Maybe the shiny new world isn’t so different from the old one after all.