Blog Posts in August 2014

A reader sent me this question:

My company will have 10GE dark fiber across our DCs with possibly OTV as the DCI. The VM team has also expressed interest in DC-to-DC vMotion (<4ms). Based on your blogs it looks like overall you don't recommend long-distance vMotion across DCI. Will the "Data Center trilogy" package be the right fit to help me better understand why?

Unfortunately, long-distance vMotion seems to be a persistent craze that peaks with a predicable period of approximately 12 months, and while it seems nothing can inoculate your peers against it, having technical arguments might help.

My good friend Tiziano complained about the fact that BGP considers next hop reachable if there’s an entry in the IP routing table even though the router cannot even ping the next hop.

That behavior is one of the fundamental aspects of IP networks: networks built with IP routing protocols rely on fate sharing between control and data planes instead of path liveliness checks.

I first met Elisa Jasinska when she had one of the coolest job titles I ever saw: Senior Packet Herder. Her current job title is almost as cool: Senior Network Toolsmith @ Netflix – obviously an ideal guest for the Software Gone Wild podcast.

In our short chat she described some of the tools she’s working on, including an adaptation of pmacct to environments with numerous BGP exit points (more details in her NANOG presentation).

Building a private cloud infrastructure tends to be a cumbersome process: even if you do it right, you oft have to deal with four to six different components: orchestration system, hypervisors, servers, storage arrays, networking infrastructure, and network services appliances.

A few days ago I had an interesting interview with Christoph Jaggi discussing the challenges, changes in mindsets and processes, and other “minor details” one must undertake to gain something from the SDDC concepts. The German version of the interview is published on Inside-IT.ch; you’ll find the English version below.

Nexus 1000V release 5.2(1)SV3(1.1) was published on August 22nd (I’m positive that has nothing to do with VMworld starting tomorrow) and I found this gem in the release notes:

Enabling BPDU guard causes the Cisco Nexus 1000V to detect these spurious BPDUs and shut down the virtual machine adapters (the origination BPDUs), thereby avoiding loops.

It took them almost three years, but we finally have BPDU guard on a layer-2 virtual switch (why does it matter). Nice!

After a week of testing, I decided to move the main ipSpace.net web site (www.ipspace.net) as well as some of the resource servicing hostnames to CloudFlare CDN. Everything should work fine, but if you experience any problems with my web site, please let me know ASAP.

Collateral benefit: ipSpace.net is now fully accessible over IPv6.

A while ago I explained why OpenFlow might be a wrong tool for some jobs, and why centralized control plane might not make sense, and quickly got misquoted as saying “controllers don’t scale”. Nothing could be further from the truth, properly architected controller-based architectures can reach enormous scale – Amazon VPC is the best possible example.

Here’s an interesting story illustrating the potential pitfalls of multi-DC deployments and the impact of data gravity on application performance.

Long long time ago on a cloudy planet far far away, a multinational organization decided to centralize their IT operations and move all workloads into a central private cloud.

SDN evangelists talking about centralized traffic engineering, flow steering or bandwidth calendaring sometimes tend to gloss over the first rule of successful traffic engineering: Know Thy Traffic.

In a world ruled by OpenFlow you’d expect the OpenFlow controller to know all the traffic; in more traditional networks we use technologies like NetFlow, sFlow or IPFIX to report the traffic statistics – but regardless of the underlying mechanism, you need a tool that will collect the statistics, aggregate them in a way that makes them usable to the network operators, report them, and potentially act on the deviations.

I’m still getting questions about layer-2 data center interconnect; it seems this particular bad idea isn’t going away any time soon. In the face of that sad reality, let’s revisit what I wrote about layer-2 DCI over VXLAN.

VXLAN hasn’t changed much since the time I explained why it’s not the right technology for long-distance VLANs.

Last week the global routing table (as seen from some perspectives) supposedly exceeded 512K routes, and weird things started to happen to some people that are using old platforms that by default support 512K IPv4 routes in the switching hardware.

I’m still wondering whether the BGP table size was the root cause of the observed outages. Cisco’s documentation (at least this document) is pretty sloppy when it comes to the fact that usually 1K = 1024, not 1000 – I’d expect the hard limit to be @ 524.288 routes … but then maybe Cisco’s hardware works with decimal arithmetic.

The Data Center Fabric Architectures update session in late June included a whole new section on Brocade’s VCS fabric and new features they added in Network OS 4.0. The edited videos have been published and cover these topics:

Carlos Mendioroz sent me a seemingly simple question: when is a BGP route invalid? My knee-jerk reaction: when the next hop is not reachable (and I’m not the only one). WRONG – BGP routes with unreachable next hop are still valid, as shown in the following printout:

In a previous blog post I explained how load sharing across LDP-controlled MPLS core works. Now let’s focus on another detail: how are the packets assigned to individual paths across the core?

2014-08-14: Additional information was added to the blog post based on comments from Nischal Sheth, Frederic Cuiller and Tiziano Tofoni. Thank you!

One of my readers sent me this question:

I am already CCIE and work as a network engineer with pretty good salary. But I think that I am losing some passion for Cisco networking and have interests in many other technologies. Currently I am very interested in Linux and Python development. Is it worth to add some Red Hat certification along CCIE or should I pursue another CCIE?

I think “should I go for CCIE or RHCE” is the wrong question.

VXLAN is becoming de-facto encapsulation standard for overlay virtual networks (at least according to industry pundits and marketing gurus working for companies with VXLAN-based products) – even Juniper Contrail, which was traditionally a pure MPLS/VPN architecture uses it.

Not so fast – Contrail is using VXLAN packet format to carry MPLS labels between hypervisors and ToR switches.

Here’s a question that bothered me for years till I finally gave up and labbed it: does ECMP load sharing work in an MPLS core? More specifically, will an LSP split into multiple LSPs?

Maxim and myself continued our STP discussion and eventually agreed that while STP might not be the best protocol out there (remember: it had to run on Z80 CPU), it’s the only standardized thing that prevents nasty forwarding loops, prompting Maxim to ask another seemingly simple question:

What's so wrong with STP, that there are STP haters out there turning it off wherever they see it?

Welcome to the wonderful world of Expert Beginners.

Someone made a really interesting remark on my Disable Console Logging blog post: Cisco IOS has log message rate limiting – all you need is the logging rate-limit configuration command.