Blog Posts in April 2019

One of my readers sent me this email after reading my Loop Avoidance in VXLAN Networks blog post:

Not much has changed really! It’s still a flood/learn bridged network, at least in parts. We count 2019 and talk a lot about “fabrics” but have 1980’s networks still.

The networking fundamentals haven’t changed in the last 40 years. We still use IP (sometimes with larger addresses and augmentations that make it harder to use and more vulnerable), stream-based transport protocol on top of that, leak addresses up and down the protocol stack, and rely on technology that was designed to run on 500 meters of thick yellow cable.

My first Cisco router was a blade for a Cabletron modular hub (anyone remembers what hubs were or a company named Cabletron?). We plugged it in, I read the documentation, figured out I had to type conf t and was faced with a blinking cursor staring back at me from an empty line.

A few years later I was invited to beta test Cisco software release 9.21 (it wasn’t called IOS yet). The best feature it had was the awesome configuration CLI with context-sensitive prompts and on-demand help.

Have you ever wondered how many free ports you have on your stackable campus switches? I’m sure there must be a wonderful network management tool that creates that reports with a click of a button… but what if the tool your PHB purchased based on awesome PowerPoint and glitzy demo can’t do that?

Nadeem Lughmani decided to solve this challenge as a hands-on assignment in the Building Network Automation Solutions online course and created an Ansible playbook and a Python plugin that counts the total number of ports and number of free ports for each switch stack specified in the device inventory.

Wonder what else course attendees created in the past? Here’s a small sample.

We all know about catastrophic headline-generating failures like AWS East-1 region falling apart or a major provider being down for a day or two. Then there are failures known only to those who care, like losing a major exchange point. However, I’m becoming more and more certain that the known failures are not even the tip of the iceberg – they seem to be the climber at the iceberg summit.

This blog post was initially sent to subscribers of my SDN and Network Automation mailing list. Subscribe here.

One of the common questions I get once the networking engineers progress from Ansible 101 to large-scale deployments (example: generating configurations for 1000 devices) is “Can Ansible use a relational database? Text files don’t scale…”

TL&DR answer: Not directly, but there are tons of database Ansible plugins or custom Jinja2 filters out there.

Remember how Nick Buraglio tried to use OpenDaylight to build a small part of SuperComputing conference network… and ended up with a programmable patch panel?

This time he repeated the experiment using Faucet SDN Controller – an OpenFlow controller focused on getting the job done – and described his experience in Episode 101 of Software Gone Wild.

We started with the usual “what problem were you trying to solve” and quickly started teasing apart the architecture and got geekily focused on interesting things like:

This is a guest blog post by Dave Crown, Lead Data Center Engineer at the State of Delaware. He can be found automating things when he's not in meetings or fighting technical debt.

In a recent blog post, Ivan postulated “You’d execute a REST API call. Any one of those calls might fail. Now what? ... You’ll have absolutely no help from the orchestration system because REST API is not transactional so there’s no rollback.” Well, that depends on the orchestration system in use.

The promise of controller-based solutions (ACI, NSX, etc.) is that your unicorn powered network controller should be an all seeing, all knowing platform managing your network. We all have hopefully learned about the importance of backups very early on our careers. Backup and, more importantly, restore should be table stakes; a fundamental feature of any network device, let alone a networking system managed by a controller imbued with magical powers (if the vendor is to be believed).

Every time I’m running a data center-related workshop I inevitably get pulled into stretched VLAN and stretched clusters discussion. While I always tell the attendees what the right way of doing this is, and explain the challenges of stretched VLANs from all perspectives (application, database, storage, routing, and broadcast domains) the sad truth is that sometimes there’s nothing you can do.

In those sad cases, I can give the workshop attendees only one advice: face the reality, and figure out how badly you might fail. It’s useless pretending that you won’t get into a split-brain scenario - redundant equipment just makes it less likely unless you over-complicated it in which case adding redundancy reduces availability. It’s also useless pretending you won’t be facing a forwarding loop.

This blog post was initially sent to subscribers of my SDN and Network Automation mailing list. Subscribe here.

I was walking down the infinite hallways of Cisco Live Europe chatting with the fellow Tech Field Day Extra delegates when I probably blanked out for a minute as the weirdest of thoughts hit me: “REST API is not transactional”

TL&DR: Apart from using structured data and having error codes REST API is functionally equivalent to Cisco IOS CLI from 1995

This is a guest blog post by Albert Siersema, senior network and cloud engineer at Mediacaster.nl. He’s always busy broadening his horizons and helping his customers in (re)designing and automating their infrastructure deployment and management.

We’d like to be able to automate our network deployment and management from a single source of truth, but before we get there from a running (enterprise, campus!) network, we’ll have to take some small steps first.

These posts are not focused on 802.1x, but it serves as a nice use case in which I’ll show you how automation can save time and bring some consistency and uniformity to the network (device) configuration.

I’m not the only one telling people not to bet the farm on Santa Claus and dancing unicorns. Pete Welcher wrote a nice blog post describing the implications of laws of physics and data gravity (I described the gory details in Designing Active-Active Data Centers and AWS Networking Deep Dive webinars).

Meanwhile, Russ White reviewed an article that (without admitting it) discovered that serverless is just software running on other people’s servers.

Enjoy!

A long while ago I published a sample Ansible/NAPALM/Jinja2 solution that would take LLDP information and turn it into a network diagram (I described its details in a short video that’s accessible to anyone attending our network automation course or having an Expert subscription).

The trickiest part of that solution was detection of bidirectional links:

When I started working with Cisco routers in late 1980s all you could get were devices with a dozen or so ports, and CPU-based forwarding (marketers would call it software defined these days). Not surprisingly, many presentations in Cisco conferences (before they were called Networkers or Cisco Live) focused on good network design and split of functionality in core, aggregation (or distribution) and access layer.

What you got following those rules were stable and predictable networks. Not everyone would listen; some customers tried to be cheap and implement too many things on the same box… with predictable results (today they would be quick to blame vendor’s poor software quality).

This blog post was initially sent to subscribers of my SDN and Network Automation mailing list. Subscribe here.

One of my readers sent me this question:

Would you write about methods for reverting from expected new state to old state in the case automation went wrong due to (un)predictable events that left a node or network in a limbo state betwixt and between.

Like always, there’s the easy and the really hard part.

Last Thursday I started another experiment: a series of live webinar sessions focused on business aspects of networking technologies. The first session expanded on the idea of three paths of enterprise IT. It covered the commoditization of IT and networking in particular, vendor landscape, various attempts at segmenting customers, and potential long-term Enterprise IT paths. Recording is already online and currently available with standard subscription.

Although the attendance was lower than usual, attendees thoroughly enjoyed it – one of them sent me this: “the value of ipSpace.net is that you cut through the BS”. Mission accomplished ;)

If you're at least vaguely familiar with modern black hole theories, you'll totally enjoy the concept of email event horizon.

A while ago I made a statement along the lines of “MPLS segment routing is the best thing that happened to MPLS control plane in a decade”. Obviously some MPLS-focused engineers disagree with that and a few years ago I decided to write a lengthy blog post explaining the differences between using MPLS SR with IGP (or BGP) versus more traditional IGP+LDP approach.

Obviously, I wasn’t making any progress on that front, so the only way forward was to record a short video on the topic which didn’t work well either because the end-result was a set of three videos (available with free or paid ipSpace.net subscription).

• Principles of MPLS-based transport core;
• Drawbacks of MPLS label distribution with LDP;
• MPLS label distribution in Segment Routing

When I started working with Ansible networking modules they had a distinct science fair feel: everything was in flux, every new version of Ansible would break my playbooks, modules would disappear from one release to next, documentation was sketchy and describing the latest development code not a shipped release.

In the meantime, code, documentation, and release/deprecation management improved dramatically:

Last year I got into somewhat-heated discussion with a few engineers who followed the advice to run IBGP EVPN address family on top of an EBGP underlay.

My main argument was simple: this is not how BGP was designed and how it’s commonly used, and twisting it this way requires a schizophrenic BGP routing process, which introduces unnecessary complexity (even though it looks simple in Junos configuration) and might confuse people who have to run the network after the brilliant designer is gone.

This is a guest blog post by Andrea Dainese, senior network and security architect, and author of UNetLab (now EVE-NG) and  Route Reflector Labs. These days you’ll find him busy automating Cisco ACI deployments.

Following the Ivan’s post about Firewall Ruleset Automation, I decided to take a step forward: can we always have up-to-date and clean firewall policies without stale rules?

The problem

We usually configure and manage firewalls using a process like this:

In April 2019 we’re starting a new cloud security saga with Matthias Luft. The first webinar in this series will focus on the basics, subsequent live sessions spread through the rest of 2019 will cover individual technologies.

Another series we’re starting is Business Aspects of Networking, opening on April 4^th with Three Paths of Enterprise IT.

We’ll also continue the math-in-networking series, this time focused on reliability functions and advanced reliability topics.