Category: fabric

Many vendors talk about network automation these days, and almost all of them gloss over an important detail: automation works best when you manage to simplify things to the bare minimum needed to get the job done.

One of the vendors that focus on simplifying the network device configuration is Cumulus Linux.

I got into an interesting discussion with a fellow networking engineer trying to understand the impact of a switch failure in a L2/L3 data center fabric (anything from Avaya’s fabric or Brocade’s VCS Fabric to Cisco’s FabricPath, ACI or Juniper’s QFabric) on MAC and ARP tables.

Here’s my take on the problem – have I missed anything?

Dinesh Dutt started his part of the Data Center Fabrics Update webinar with “what is Cumulus Linux all about” and “what data center architectures does it support” and then quickly jumped into details about the base technologies used by Cumulus Linux: MLAG and IP routing.

Not surprisingly, the MLAG part generated tons of questions, and Dinesh answered all of them, even when he had to say “We don’t do that.”

After introducing the concepts of Cumulus Linux in the Data Center Fabrics update session, Dinesh Dutt described the typical data center architectures implemented with Cumulus Linux and the lessons everyone should learn from large-scale web properties.

With the advent of layer-3 leaf-and-spine data center fabrics, it became (almost) possible to build pure layer-3-only data center networks… if only the networking vendors would do the very last step and make every server-to-ToR interface a layer-3 interface. Cumulus decided to do just that.

In May 2015 I invited Dinesh Dutt to talk about Cumulus Linux and its typical use cases on an update session of the Data Center Fabrics Architecture webinar.

As expected, he started with the big picture: what are Cumulus Networks and Cumulus Linux all about?

During the Cumulus Linux presentation Dinesh Dutt had at Data Center Fabrics webinar, someone asked an unexpected question: “Do you have In-Service Software Upgrade (ISSU) on Cumulus Linux” and we both went like “What? Why?”

Dinesh is an honest engineer and answered: “No, we don’t do it” with absolutely no hesitation, but we both kept wondering, “Why exactly would you want to do that?”

Packet Pushers podcast is a constant source of inspiration for my blog posts. Recently I stumbled upon Rob Sherwood’s explanation of how they package Big Cloud Fabric:

It’s a vertically integrated solution, from Switch Light OS to our SDN controller and Big Cloud Fabric application.

Really? What happened to openness and disaggregation?

I was running the first part of the Data Center Fabrics Update webinar last week, mentioned that Brocade VDX 6740 supports Flex ports (a port you can use as Fibre Channel or 10GE port), and someone immediately wrote a comment saying “so does VDX 6940”. I was almost sure Flex ports aren’t available on VDX 6940 yet, and as always turned to vendor documentation to figure it out.

As expected, the data sheet is a bit vague, somewhat reflecting reality, but also veering into the realm of futures instead of features. Here’s what they say:

One of the responses I got on my “What is Layer-2” post was

Ivan, are you saying to use L3 switches everywhere with /31 on the switch ports and the servers/workstation?

While that solution would work (and I know a few people who are using it with reasonable success), it’s nothing more than creative use of existing routing paradigms; we need something better.

In mid-February a blog post on Cisco’s web site announced stretched ACI fabric (bonus points for not using marketing grammar but talking about a shipping product). Will it work better than other PowerPoint-based fabrics? You bet!

What’s the Big Deal?

Cisco’s ACI fabric uses distributed (per-switch) control plane with APIC controllers providing fabric configuration and management functionality. In that respect, the ACI fabric is no different from any other routed network, and we know that those work well in distributed environments.

My good friend Tom Hollingsworth wrote a great blog post about hypermyopia in the networking industry. I agree with most everything he wrote (I have to – I’m always telling people to focus on business needs and to change their mentality before relying on shiny new gizmos), but I still think it’s crucial to consider the technology used in products we’re looking at.

Late last year David Gee and I wanted to test another interesting gizmo: an online virtual whiteboard. David was pondering some interesting aspect of Cisco ACI and they seemed like a perfect topic for an impromptu discussion.

I’m still convinced that architectures with centralized control planes (and that includes solutions relying on OpenFlow controllers) cannot scale. On the other hand, Big Switch Networks is shipping Big Cloud Fabric, and they claim they solved the problem. Obviously I wanted to figure out what’s going on and Andy Shaw and Rob Sherwood were kind enough to explain the interesting details of their solution.

Long story short: Big Switch Networks significantly extended OpenFlow.

Listening to some SDN pundits one gets an impression that SDN brings peace to Earth, solves all networking problems and makes networking engineers obsolete.

Cynical jokes aside, and ignoring inevitable bugs, is controller-based networking really more reliable than what we do today?