Building network automation solutions

9 module online course

Start now!

Blog Posts in January 2023

Design Clinic: Small-Site IPv6 Multihoming

I decided to stop caring about IPv6 when the protocol became old enough to buy its own beer (now even in US), but its second-system effects keep coming back to haunt us. Here’s a question I got for the February 2023 ipSpace.net Design Clinic:

How can we do IPv6 networking in a small/medium enterprise if we’re using multiple ISPs and don’t have our own IPv6 Provider Independent IPv6 allocation. I’ve brainstormed this with people far more knowledgeable than me on IPv6, and listened to IPv6 Buzz episodes discussing it, but I still can’t figure it out.

read more see 1 comments

netlab Release 1.5.0: Larger Lab Topologies

netlab release 1.5.0 includes features that will help you start very large lab topologies (someone managed to run over 90 Mikrotik routers on a 24-core server):

To get more details and learn about additional features included in release 1.5.0, read the release notes. To upgrade, execute pip3 install --upgrade networklab.

New to netlab? Start with the Getting Started document and the installation guide.

add comment

MUST READ: Nothing Works

Did you ever wonder why it’s impossible to find good service company, why most software sucks, or why networking vendors can get away with selling crap? If you did, and found no good answer (apart from Sturgeon’s Law), it’s time to read Why is it so hard to buy things that work well? by Dan Luu.

Totally off-topic: his web site uses almost no CSS and looks in my browser like a relic of 1980s. Suggestions how to fix that (in Chrome) are most welcome.

see 2 comments

Hiding Malicious Packets Behind LLC SNAP Header

A random tweet1 pointed me to Vulnerability Note VU#855201 that documents four vulnerabilities exploiting a weird combination of LLC and VLAN headers can bypass layer-2 security on most network devices.

Before anyone starts jumping up and down – even though the VLAN header is mentioned, this is NOT VLAN hopping.

The security researcher who found the vulnerability also provided an excellent in-depth description focused on the way operating systems like Linux and Windows handle LLC-encapsulated IP packets. Here’s the CliffNotes version focused more on the hardware switches. Even though I tried to keep it simple, you might want to read the History of Ethernet Encapsulation before moving on.

read more add comment

Response: Network Automation Expert Beginners

I usually post links to my blog posts to LinkedIn, and often get extraordinary comments. Unfortunately, those comments usually get lost in the mists of social media fog after a few weeks, so I’m trying to save them by reposting them as blog posts (always with original author’s permission). Here’s a comment David Sun left on my Network Automation Expert Beginners blog post


The most successful automation I’ve seen comes from orgs who start with proper software requirements specifications and more importantly, the proper organizational/leadership backing to document and support said infrastructure automation tooling.

read more add comment

Will DPUs Change the Network?

It’s easy to get excited about what seems to be a new technology and conclude that it will forever change the way we do things. For example, I’ve seen claims that SmartNICs (also known as Data Processing Units – DPU) will forever change the network.

TL&DR: Of course they won’t.

Before we start discussing the details, it’s worth remembering what a DPU is: it’s another server with its own CPU, memory, and network interface card (NIC) that happens to have PCI hardware that emulates the host interface cards. It might also have dedicated FPGA or ASICs.

read more add comment

netlab: Building a Layer-2 Fabric

A friend of mine decided to use netlab to build a simple traditional data center fabric, and asked me a question along these lines:

How do I make all the ports be L2 by default i.e. not have IP address assigned to them?

Trying to answer his question way too late in the evening (I know, I shouldn’t be doing that), I focused on the “no IP addresses” part. To get there, you have to use the l2only pool or disable IPv4 prefixes in the built-in address pools, for example:

read more add comment

Worth Reading: Do We Need Network Automation

A long, long time ago, Mircea Ulinic (the author of Salt networking modules) wrote a long and thoughtful blog post on whether we need network automation (TL&DR spoiler: yes).

After reading the article, you might want to listen to the Salt and SaltStack podcast we did with Mircea a long while ago, and watch his presentation in Building Network Automation Solutions online course (also accessible with Expert Subscription).

see 1 comments

Relationships between Layer-2 (VLAN) and Layer-3 (Subnet) Segments

Sometimes it takes me years to answer interesting questions, like the one I got in a tweet in 2021:

Do you have a good article describing the one-to-one relation of layer-2 and layer-3 networks? Why should every VLAN contain one single L3 segment?

There is no mandatory relationship between multi-access layer-2 networks and layer-3 segments, and secondary IP addresses (and subnets) were available in Cisco IOS in early 1990s. The rules-of-thumb1 claiming there should be a 1:1 relationship usually derive from the oft-forgotten underlying requirements. Let’s start with those.

read more see 4 comments

Feedback: Docker Networking Deep Dive

While the pundits keeps telling me Docker is dead (looking at its documentation I would say they’re right) and Kubernetes it the way to go (yay!), some people still have to deal with Docker networking, and at least some of them found the Docker Networking Deep Dive webinar useful. Here’s a recent review:

You can scroll over internet pages as long as you can, you will rarely find this kind of specialized knowledge. This is the next level in term of knowledge about Docker.

If you belong to the “Kubernetes will rule the world” camp, we have you covered as well: Stuart Charlton created a phenomenal Kubernetes Networking Deep Dive webinar (approximately half of it is already accessible with free subscription).

add comment

Network Automation Expert Beginners

Some network automation skeptics came to that place the hard way: they got burned by half-baked semi-tested systems. This is what one of my good friends had to say in a LinkedIn comment:

I am suspicious of automation, as I’ve unfortunately seen too many outages caused by either human error or faulty automation. Every time it required human CLI/GUI intervention to correct it. The problem is that the more automation we push, the fewer people know how to use the “old school” way to administer stuff.

Network automation is not the only IT discipline that could cause hard-to-correct errors requiring manual intervention. I’m positive everyone knows at least one horror story resulting in manual tweaking of the Windows registry, or a sequence of arcane SQL commands1.

read more see 2 comments

netlab Release 1.4.3: Cisco IOS XRv, MPLS on FRR

I had tons of plans to implement new netlab features during the last week of December, but then (fortunately) reality intervened and I spent my time relaxing and enjoying the break. I still managed to add IOS XRv support to netlab release 1.4.3 though ;). Other new features include:

To upgrade, execute pip3 install --upgrade networklab.

New to netlab? Start with the Getting Started document and the installation guide.

add comment

Worth Reading: Routing Protocol Implementation Evaluation

In 2018 I tried to figure out whether the rush to deploy new routing protocols in leaf-and-spine fabrics is anything more than another blob of hype (RIFT, OpenFabric, BGP), considering OSPF got the job done for AWS. Those discussions probably sounded like a bunch of smart kids trying to measure outside temperature with a moist finger, so the only recommendation I could give in 2021 was “use the best tool for the job, keeping in mind you’re not Google or Microsoft

It’s always better to measure than to have opinions, and a group of academics did just that. They developed Sybil – a tool to measure routing protocol performance in leaf-and-spine fabrics – and Dip Singh used it to compare BGP to IS-IS and OpenFabric.

add comment

Video: MLAG with EVPN Deep Dive

In November 2022 I described some of the intricacies of using EVPN to implement MLAG control plane. You might have noticed that I didn’t dive deep into EVPN details, and I had a good reason for that – Lukas Krattiger did a wonderful job describing how MLAG works with EVPN in the EVPN Deep Dive webinar.

You need Free ipSpace.net Subscription to watch the video. To watch the whole webinar, buy Standard or Expert ipSpace.net Subscription.
add comment

State of LDPv6 and 6PE

One of my readers successfully deployed LDPv6 in their production network:

We are using LDPv6 since we started using MPLS with IPv6 because I was used to OSPF/OSPFv3 in dual-stack deployments, and it simply worked.

Not everyone seems to be sharing his enthusiasm:

Now some consultants tell me that they know no-one else that is using LDPv6. According to them “everyone” is using 6PE and the future of LDPv6 is not certain.

read more add comment

Updates: AWS Networking

It didn’t make sense to update Amazon Web Services Networking webinar before the re:Invent conference – even though AWS introduced only a few networking features during the conference, at least one of them made a significant impact on the materials.

However, once the conference was over, I went over the to-do list that has been slowly accumulating for months and spent days updating over a dozen videos1. The major changes include:

read more add comment

Is It Time to Replace TCP in Data Centers?

One of my readers asked for my opinion about the provocative “It’s Time to Replace TCP in the Datacenter” article by prof. John Ousterhout. I started reading it, found too many things that didn’t make sense, and decided to ignore it as another attempt of a proverbial physicist solving hard problems in someone else’s field.

However, pointers to that article kept popping up, and I eventually realized it was a position paper in a long-term process that included conference talks, interviews and keynote speeches, so I decided to take another look at the technical details.

read more see 4 comments

netlab Release 1.4.2: Juniper vMX and Junos Features

One of the last things I did before going on the Christmas break was to push out netlab release 1.4.2. Its highlights include:

  • Juniper vMX by Stefano Sasso
  • BFD, VRF, MPLS, SR-MPLS, and MPLS/VPN on Junos (also by Stefano)
  • Full VLAN support on vMX and routed VLAN interfaces on vSRX (yet again, Stefano’s contribution)
  • VyOS containerlab support by Oleg A. Arkhangelsky
  • CSR 1000v VLAN and VXLAN support

Upgrading is as easy as ever: execute pip3 install --upgrade networklab.

New to netlab? Start with the Getting Started document and the installation guide.

add comment
Sidebar