Example: Multi-Stage Clos Fabrics

Smaller Clos fabrics are built with two layers of switches: leaf and spine switches. The oversubscription ratio you want to achieve dictates the number of uplinks on the leaf switch, which in turn dictates the maximum number of spine switches and thus the fabric size.

You have to use multi-stage Clos architecture if you want to build bigger fabrics; Brad Hedlund described a sample fabric with over 24.000 server-facing ports in the Clos Fabrics Explained webinar.

4 comments:

  1. How can you support 128 spine switches if most routing protocols only provide 8 or 16 max paths? This is a big disconnect in my understand of Clos fabric implementations.

    ReplyDelete
  2. Why would you need 128 spine switches? You can have as many spine switches as you have uplinks on leaf switches - and many DC switches support 32-way ECMP.

    ReplyDelete
  3. In the video he show 64 stage 3 spine switches connected to 128 stage 2 spine switches. That is the part I don't understand. Even if as you say they can support 32-way ECMP (which vendors support this?), that still doesn't account for that many switches.

    ReplyDelete
  4. Hi Ivan, I don't get the math either. At leaf switches (stage 2), you have 4*40GE southbound, towards the access (ToR); as the fabric should be non-blocking, you need the same capacity northbound (to stage 3 - spine). So you can use either 4*40GE (4 spine switches/32 leaf switches) or, to support more leaf switches, 16*10GE (16*spine/128 leaf). Why are there 64 spine switches in the presentation?
    Thanks for the explanation.

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.