I had an interesting conversation with Doug Hanks (@douglashanksjr) about the need for intra-spine links in leaf-and-spine fabric designs. You clearly don’t need links between spine switches when every leaf node (switch or router/firewall/load balancer) is connected to all spine switches ... but what happens when one of the leaf-to-spine links fails? Will other leaf switches know that they have to avoid the spine switch with the failed link?

Consider the leaf-and-spine fabric in the following diagram. Would leaf switch Z know it shall not send traffic to leaf switch A through spine switch C if the A-C link fails?

If the answer is YES, you don’t need intra-spine links (at least not for user traffic). If the answer is NO, you better have them.

There are at least two scenarios where the leaf switches wouldn’t have complete visibility into the fabric topology (and could send the traffic to the wrong spine switch)

• When you use multi-chassis link aggregation between leaf and spine layers (in which case the vendor design rules always mandate intra-spine links anyway);
• When you’re doing route summarization between spine and leaf layer in pure L3 fabrics.

Finally, there’s the multiple failures scenario: let’s say you have two spine switches (S1 and S2) and leaf switch A loses connectivity to S1 while leaf switch B loses connectivity to S2. You wouldn’t want the traffic from A to B to go through a third leaf switch, would you?

More details

I’ve covered (almost) every possible leaf-and-spine design scenario (for L2-only, L3-only and mixed fabrics) in the Clos Fabrics Explained webinar.