Category: data center

Here’s an interesting story illustrating the potential pitfalls of multi-DC deployments and the impact of data gravity on application performance.

Long long time ago on a cloudy planet far far away, a multinational organization decided to centralize their IT operations and move all workloads into a central private cloud.

I’m still getting questions about layer-2 data center interconnect; it seems this particular bad idea isn’t going away any time soon. In the face of that sad reality, let’s revisit what I wrote about layer-2 DCI over VXLAN.

VXLAN hasn’t changed much since the time I explained why it’s not the right technology for long-distance VLANs.

The Data Center Fabric Architectures update session in late June included a whole new section on Brocade’s VCS fabric and new features they added in Network OS 4.0. The edited videos have been published and cover these topics:

When I published the Data Center Design Case Studies book almost exactly a month ago, three chapters were still missing – but that was the only way to stop the procrastination and ensure I’ll write them (I’m trying to stick to published deadlines ;).

The first one of the missing chapters is already finished and available to subscribersand everyone who bought the book or Designing Private Cloud Infrastructure webinar (you’ll also get a mailing on Sunday to remind you to download the fresh copy of the PDF).

The Amazon Kindle version will be updated in a few days.

The last generations of high-end servers are amazing: they can have terabyte (or more) of RAM, dozens of CPU cores, and four (or more) 10GE uplinks. It’s easy to pack 100+ well-behaved VMs on a single server, reducing the whole data center into a private cloud that fits into a single rack.

If you’re an avid Kindle user, you can buy the Data Center Design Case Studies book on Amazon. Here’s the Amazon.com link, search should find it on most other Amazon marketplaces.

However, if you can survive reading the PDF version, please buy it straight off my web site. Here’s why:

After the excellent IPv6 security presentation Eric Vyncke had @ 9th Slovenian IPv6 summit someone asked me: “Why is IPv6 first-hop security so complex? It looks like the developers of IPv6 protocol stack tried to make users anonymous and made everyone’s life complex while doing that.”

Well, he was totally surprised by my answer: “The real reason IPv6 first-hop security is so complex is the total mess we made of L2/L3 boundary.”

Most recently launched data center switches use the Trident 2 chipset, and yet we know almost nothing about its capabilities and limitations. It might not work at linerate, it might have L3 lookup challenges when faced with L2 tunnels, there might be other unpleasant surprises… but we don’t know what they are, because you cannot get Broadcom’s documentation unless you work for a vendor who signed an NDA.

Interestingly, the best source of Trident 2 technical information I found so far happens to be the Cisco Live Nexus 9000 Series Switch Architecture presentation (BRKARC-2222). Here are a few tidbits I got from that presentation and Broadcom’s so-called datasheet.

One of my readers sent me an interesting question:

I have been reading at many places about "throwing more bandwidth at the problem." How far is this statement valid? Should the applications(servers) work with the assumption that there is infinite bandwidth provided at the fabric level?

Moore’s law works in our favor. It’s already cheaper (in some environments) to add bandwidth than to deploy QoS.

During yesterday’s Data Center Fabrics Update presentation, one of the attendees sent me this question while I was talking about the Arista 7300 series switches:

Is the 7300 really non-blocking at all packet sizes? With only 2 x Trident-2 per line card it can't support non-blocking for small packets based on Trident-2 architecture.

It was an obvious example of vendor bickering, so I ignored the question during the presentation, but it still intrigued me, so I decided to do some more research.

A long while ago there was an interesting discussion started by Brad Hedlund (then at Dell Force10) comparing leaf-and-spine (Clos) fabrics built from fixed-configuration pizza box switches with high-end chassis switches. The comments made by other readers were all over the place (addressing pricing, wiring, power consumption) but surprisingly nobody addressed the queuing issues.

This blog post focuses on queuing mechanisms available within a switch; the next one will address end-to-end queuing issues in leaf-and-spine fabrics.

HP representatives made some pretty bold claims during Networking Tech Field Day 1, including “our switches will support EVB, FCoE, SPB and TRILL.” I took them three years to deliver on those promises (and the hardware they had at that time doesn’t exactly support all features they promised), but their current protocol coverage is impressive.

Good news: In the last few months, almost all major data center Ethernet switching vendors (Arista, Cisco, Dell Force 10, HP, and Juniper) released documented GA version of OpenFlow on some of their data center switches.

Bad news: no two vendors have even remotely comparable functionality.

One of my readers sent me this question:

I have a data center with huge L2 domains. I would like to move routing down to the top of the rack, however I’m stuck with a load-balancing question: how do load-balancers work if you have routed network and pool members that are multiple hops away? How is that possible to use with Direct Return?

There are multiple ways to make load balancers work across multiple subnets:

Craig Matsumoto recently quoted some astonishing claims from Dell’Oro Group analyst Alan Weckel:

• Whitebox switches (combined) will be the second largest ToR vendor;
• Whitebox 10GE ports will cost around $100.

Let’s try to guestimate how realistic these claims are.