Category: design

The proponents of the “let’s run EVPN over EBGP underlay” idea often ignore an interesting challenge: EVPN advocates the use of automatically-generated Route Targets, which might not work when every leaf switch uses a different AS number.

I explored this particular can of worms in the EVPN Route Target Considerations section of the Using BGP in a Data Center Leaf-and-Spine Fabric saga.

My BGP in EVPN-Based Data Center Fabrics blog post generated numerous comments from engineers disagreeing with my views on using IBGP-over-EBGP.

As usual, there were three kinds of comments:

We managed to get another awesome lineup of speakers for the Spring 2018 Building Next-Generation Data Center online course.

Russ White, one of the authors of CCDE and CCAr programs and highly respected book author will start the course with a topic everyone should always consider when designing new infrastructure: how do you identify tradeoffs and manage complexity, making sure you meet the customer requirements while at the same time having an easy-to-operate infrastructure.

Russ White wrote a great article along the lines of what we discussed a while ago. My favorite part:

There are companies who consider the network an asset, and companies that consider the network a necessary evil.

Enjoy!

On a tangential topic: Russ will talk about network complexity in the Building Next-Generation Data Center online course starting on April 25^th.

Years ago, I decided to try out another idea: solving real-life challenges with the help of an easy-to-consume online consulting service. When I discussed the idea with my friends during one of the early Networking Field Day events the opinion was pretty unanimous: “this will never work”

Fortunately, they were wrong. Not only did ~100 customers decided to use it in the meantime, the simple idea grew to a point where I couldn’t do it all on my own.

I had an interesting “how do you build a small fabric without throwing every technology in the mix” discussion with Nicola Modena and mentioned that I don’t see a reason to use EVPN in fabrics with just a few switches. He disagreed and gave me a few good scenarios where EVPN might be handy. Before discussing them let’s establish a baseline.

The Setup

Assume you’re building two small data center fabrics (small because you have only a few hundred VMs and two because of redundancy and IT auditors).

Although it’s almost three months till the start of the Building Next-Generation Data Center online course, we already have most of the guest speakers. Today I’d like to introduce the first two (although they need no introduction).

You might have heard about Russ White. He was known as Mr. CCDE when that program started and recently focused more on data centers, open networking and whitebox switching. He’s also an authority on good network design and architecture, network complexity, and tradeoffs you have to make when designing a network.

Regardless of how much I write about (the ridiculousness of using) stretched VLANs, I keep getting questions along the same lines. This time it’s:

What type of applications require L2 Extension and L3 extension?

I don’t think I’ve seen anyone use L3 extension (after all, isn’t that what Internet is all about), so let’s focus on the first one.

Stretched VLANs (or L2 extensions) are used to solve a number of unrelated problems, because once a vendor sold you a hammer everything starts looking like a nail, and once you get used to replacing everything with nails, you want to use them in all possible environments, including public and hybrid clouds.

Two weeks ago we discussed whether it makes sense to use BGP as the routing protocol in a data center fabric. Today we’ll tackle three additional design challenges:

• Should you use IBGP or EBGP?
• When should you run BGP on the spine switches?
• Should every leaf switch have a different AS number or should they share the same AS number?

A while ago I helped a large enterprise redesign their data center fabric. They did a wonderful job optimizing their infrastructure, so all they really needed were two switches in each location.

Some vendors couldn’t fathom that. One of them proposed to build a “future-proof” (and twice as expensive) leaf-and-spine fabric with two leaves and two spines. On top of that they proposed to use EBGP as the only routing protocol because draft-lapukhov-bgp-routing-large-dc – a clear case of missing the customer needs.

One of my friends wanted to design a nice-and-easy layer-3 leaf-and-spine fabric for a new data center, and got blindsided by a hyperconverged vendor. Here’s what he wrote:

We wanted to have a spine/leaf L3 topology for an NSX deployment but can’t do that because the Nutanix servers require L2 between their nodes so they can be in the same cluster.

I wanted to check his claims, but Nutanix doesn’t publish their documentation (I would consider that a red flag), so I’m assuming he’s right until someone proves otherwise (note: whitepaper is not a proof of anything ;).

I published the last part of my Optimize Data Center Infrastructure series: build an optimized data center fabric.

To learn more about data center fabric designs, check the new online course or enroll into the Spring 2018 session of Building Next-Generation Data Center course.

One of my readers sent me this question:

I'm having an internal debate whether to use firewall-based VPNs or DMVPN to connect several sites if our MPLS connection goes down. How would you handle it? Do you have specific courses answering this question?

As always, the correct answer is it depends, in this case on:

Everyone knows that Service Providers and Enterprise networks diverged decades ago. More precisely, organizations that offer network connectivity as their core business usually (but not always) behave differently from organizations that use networking to support their core business.

Obviously, there are grey areas: from people claiming to be service providers who can’t get their act together, to departments (or whole organizations) who run enterprise networks that look a lot like traditional service provider networks because they’re effectively an internal service provider.

One of the design scenarios we covered in Leaf-and-Spine Fabric Architectures webinar is a pure layer-3 data center, and in the “how do I do this” part of that section Dinesh Dutt talked about the details you need to know to get this idea implemented on Cumulus Linux.

We covered a half-dozen design scenarios in that webinar; for an even wider picture check out the new Designing and Building Data Center Fabrics online course.