Category: data center

I wrote (and spoke) at length about layer-2 and layer-3 gateways between VLANs and overlay virtual networks, but I still get questions along the lines of “how will you connect legacy servers to the new cloud infrastructure that uses VXLAN?”

A networking engineer was trying to persuade me of importance of hardware VXLAN VTEPs. We quickly agreed physical-to-virtual gateways are the primary use case, and he tried to illustrate his point by saying “Imagine you have 1000 servers in your data center and you manage to virtualize 80% of them. How will you connect them to the other 200?” to which I replied, “That doesn’t make any sense.” Here’s why.

One of my readers sent me an interesting question:

How does one impose a security policy on servers connected via a Clos fabric? The traditional model of segregating servers into vlans/zones and enforcing policy with a security device doesn’t fit here. Can VRF-lite be used on the mesh to accomplish segregation?

Good news: the security aspects of leaf-and-spine fabrics are no different from more traditional architectures.

Network designers (and smart consulting and system integration companies) often use ExpertExpress to get a second opinion on a design someone put together using technologies they’re not thoroughly familiar with. Not surprisingly, some of those third-party designs aren’t exactly optimal.

A while ago I was asked to review a data center “design” proposed to my customer by a system integrator. It had a pair of Nexus 5500 switches connecting servers and storage to a single Nexus 7000, which was then connected to WAN edge routers.

I had a nice chat with Doug Gourlay from Arista during the Interop Las Vegas and he made an interesting remark along the lines of “in leaf-and-spine fabrics it doesn’t make sense to use redundant supervisors in switches – they cause more problems than they solve.”

As always, in the end it all depends on your environment and use case, but he definitely has a point; good engineering always works better than a heap of kludges.

A few years ago I lambasted the lack of STP support in Brocade’s VCS fabric. It took Brocade over two years to solve the problem, but they finally came up with an interesting end-to-end solution.

Here are a few highlights; for more details read the Configuring STP-type Protocols section in Network OS Administrator Guide.

Last June Tore Anderson talked about his IPv6-only data center deployment (the idea made very popular recently after Facebook’s presentation @ V6 World Congress) in one of my free webinars. In case you missed the videos explaining the technical details, watch them or view Tore’s slide deck.

A comment by Pieter E. Smit on my vSphere Does Not Need LAG Bandaids post opened yet another can of worms: vSphere behavior on uplink recovery.

Short summary: vSphere starts using an uplink as soon as its physical layer becomes operational, which might happen during ToR switch startup phase, or before a ToR switch port enters forwarding state.

Whenever I mention the idea of IPv6-only data centers, I get the usual question: “Sounds great, but is anyone actually using it?” So far, my answer was: “Yeah, I know a great guy in Norway that runs this in production” As of last week, the answer is way more persuasive: “Facebook is almost there.”

A reader left the following comment on my Does Multipath TCP Matter blog post: “Why would I use MP-TCP in a data center? Couldn’t you use packet spraying at each hop and take care of re-ordering at the destination?”

Short answer: You could, but you might not want to.

I heard the following pretty bold statement while listening to an episode of my favorite podcast: “Bringing MPLS into the data center is impractical because MPLS requires custom silicon.” Really? How about checking the Intel FM 6000 product brief first?

Broadcom Trident chipset supposedly also supports MPLS. I couldn’t verify that because Broadcom considers the capabilities of their hardware highly confidential (but if you know more, do write a comment). Absolutely refreshing for a chipset that you get in almost every ToR switch you buy.

Ed Horley opened another juicy can of worms in a comment to my First-Hop Load Balancing in IPv6 post: can we use IPv6 RA for fast failover (and high availability)?

TL&DR summary: it depends.

One of my regular readers sent me a long list of FCoE-related questions:

I wanted to get your thoughts around another topic – iSCSI vs. FCoE? Are there merits and business cases to moving to FCoE? Does FCoE deliver better performance in the end? Does FCoE make things easier or more complex?

He also made a very relevant remark: “Vendors that can support FCoE promote this over iSCSI; those that don’t have an FCoE solution say they aren’t seeing any growth in this area to warrant developing a solution”.

A week after the disastrous sleet that kicked whole regions of Slovenia off power grid the servicemen of the local power distribution company (working literally days and nights) managed to restore electricity to the closest town … but it still might take days or even weeks before everyone gets it. One of the reasons: huge failure domains.

You wouldn’t believe what your second most pressing problem is when you lose electricity for a few days in the middle of a winter storm: freezer. Being a good engineer focused on redundant solutions, I bought a diesel generator before moving into the hills to keep the freezer at a reasonably low temperature in case of a long-term power loss.

I also thought about using the same generator to run our central heating. As always, I found a huge disconnect between theory and practice.