Intra-Spine Links in Leaf-and-Spine Fabrics

I had an interesting conversation with Doug Hanks (@douglashanksjr) about the need for intra-spine links in leaf-and-spine fabric designs. You clearly don’t need links between spine switches when every leaf node (switch or router/firewall/load balancer) is connected to all spine switches ... but what happens when one of the leaf-to-spine links fails? Will other leaf switches know that they have to avoid the spine switch with the failed link?

Consider the leaf-and-spine fabric in the following diagram. Would leaf switch Z know it shall not send traffic to leaf switch A through spine switch C if the A-C link fails?

If the answer is YES, you don’t need intra-spine links (at least not for user traffic). If the answer is NO, you better have them.

There are at least two scenarios where the leaf switches wouldn’t have complete visibility into the fabric topology (and could send the traffic to the wrong spine switch)

  • When you use multi-chassis link aggregation between leaf and spine layers (in which case the vendor design rules always mandate intra-spine links anyway);
  • When you’re doing route summarization between spine and leaf layer in pure L3 fabrics.

Finally, there’s the multiple failures scenario: let’s say you have two spine switches (S1 and S2) and leaf switch A loses connectivity to S1 while leaf switch B loses connectivity to S2. You wouldn’t want the traffic from A to B to go through a third leaf switch, would you?

More details

I’ve covered (almost) every possible leaf-and-spine design scenario (for L2-only, L3-only and mixed fabrics) in the Clos Fabrics Explained webinar.

4 comments:

  1. This becomes an interesting problem to solve when the spine and leaf has a unified control plane and uses Dijkstra's algorithm for forwarding :-)

    ReplyDelete
    Replies
    1. Most interesting case would be re-balancing traffic in multi-stage Clos upon a link failure (using centralized control agent) :)

      Delete
  2. Thanks a lot. It's always good to see another approach from the core/aggregation layer. I'm curious to know if there is any large scale adoption of configurations like in your first picture. I'd think...not.

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.