The early October 2021 Facebook outage generated a predictable phenomenon – couch epidemiologists became experts in little-known Bridging the Gap Protocol (BGP), including its Introvert and Extrovert variants. Unfortunately, I also witnessed several unexpected trips to Mount Stupid by people who should have known better.

To set the record straight: everyone’s been there, and the more vocal you tend to be on social media (including mailing lists), the more probable it is that you’ll take a wrong turn and end there. What matters is how gracefully you descend and what you’ve learned on the way back.

When I started creating the How Networks Really Work series, I wondered whether our subscribers (mostly seasoned networking engineers) would find it useful. Turns out at least some of them do; this is what a long-time subscriber sent me:

How Networks Really Work is great, it’s like looking from a plane and seeing how all the roads are connected to each other. I know networking just enough to design and manage a corporate network, but there are many things I have learned, used and forgotten along the way.

So, getting a broad vision helps me remember why I chose something and maybe solve my bad choices. There are many things that I may never use, but with the movement of all things in the cloud it’s great to know, or at least understand, how things really work.

Two weeks ago I replied to a battle-scar reaction to 7-layer OSI model, this time I’ll address a much more nuanced view from Russ White. Please read his article first (as always, it’s well worth reading) and when you come back we’ll focus on this claim:

The OSI Model does not accurately describe networks.

Like with any tool in your toolbox, you can view the 7-layer OSI model in a number of ways. In the case of OSI model, it can be used:

I should have known better, but I got pulled into another stretched VLANs for disaster recovery tweetfest. Surprisingly, most of the tweets were along the lines of you really shouldn’t be doing that and that would never work well, but then I guess I was only exposed to a small curated bubble of common sense… until this gem appeared in my timeline:

Interestingly, that’s exactly how IP works:

One of my readers sent me this interesting question:

It begs the question in how far graduated students with a degree in computer science or applied IT infrastructure courses (on university or college level or equivalent) are actually aware of networking fundamentals. I work for a vendor independent networking firm and a lot of my new colleagues are college graduates. Positively, they are very well versed in automation, scripting and other programming skills, but I never asked them what actually happens when a packet traverses a network. I wonder what the result would be…

I can tell you what the result would be in my days: blank stares and confusion. I “enjoyed” a half-year course in computer networking that focused exclusively on history of networking and academic view of layering, and whatever I know about networking I learned after finishing my studies.

I was speaking with a participant of an SDN event in Zurich after the presentations, and he made an interesting comment: whenever he experienced serious troubleshooting problems in his career, it was due to lack of understanding of networking fundamentals.

Let me give you a few examples: Do you know how ARP works? What is proxy ARP? How does TCP offload work and why is it useful? What is an Ethernet collision and when would you see one? Why do we need MLD in IPv6 neighbor discovery?

Writing tests that check the correctness of network device configurations is hard (overview, more details). It’s also an interesting exercise in getting the timing just right:

• Routing protocols are an eventually-consistent distributed system, and things eventually appear in the right place (if you got the configurations right), but you never know when exactly that will happen.
• You can therefore set some reasonable upper bounds on when things should happen, and declare failure if the timeouts are exceeded. Even then, you’ll get false positives (as in: the test is telling you the configurations are incorrect, when it’s just a device having a bad hair day).

And just when you think you nailed it, you encounter a device that blows your assumptions out of the water.

netlab release 25.07 was published yesterday. The major new features include:

• The ospf.areas plugin supports OSPFv2 and OSPFv3 stub areas, NSSA areas, and area ranges.
• The BGP routing policies plugin supports aggregate BGP routes
• The BGP configuration module supports BGP confederations

But wait, there’s much more:

As I was running the netlab pre-release integration tests, I noticed that ArubaCX failed the IPv6 Common Services test (it worked before). Here’s the gist of what that test does:

• It creates three VRFs (red, blue, and common)
• It imports routes from red and blue VRF into the common VRF and routes from the common VRF into the red and blue VRF (the schoolbook example of common services VRF)
• Just to be on the safe side, it imports red routes into the red VRF and so on.

Here’s the relevant part of the netlab lab topology:

Did you ever wonder why pressing an up-arrow in a (Linux) terminal window sometimes recalls the previous command but other times creates ^[[A?

Julia Evans did, and spent months exploring the quirks of the Linux terminal (and writing blog posts describing what she found), finally resulting in The Secret Rules of the Terminal (including the various shells, terminal emulators, escape codes, and TTY driver). A must-read if you’re a newbie who wants to understand why things happen the way they do.

One of the happy netlab users sent me an interesting challenge:

• He’s built a large lab and added tons of extra configuration to the lab devices.
• Afterwards, he realized he’d like to add a few more devices to the lab and was worried about losing all the changes he had made.

Unfortunately, you cannot add new devices to an already-running lab. You must shut down the lab, change the topology description, and start a new lab. However, there are things you can do to preserve the extra work you already did:

Martin Fowler published an interesting article about Expert Generalists. Straight from the abstract:

As computer systems get more sophisticated we’ve seen a growing trend to value deep specialists. But we’ve found that our most effective colleagues have a skill in spanning many specialties.

Also:

There are two sides to real expertise. The first is the familiar depth: a detailed command of one domain’s inner workings. The second, crucial in our fast-moving field is the ability to learn quickly, spot the fundamentals that run beneath shifting tools and trends, and apply them wherever we land.

Remember how I told you to focus on the fundamentals? 😎

Have you ever managed to type reload in the wrong terminal window and brought down a core switch (I probably did)? I managed to do the Ubuntu equivalent of that stupidity: I told my main Ubuntu server to sudo poweroff instead of doing that to a Vagrant VM.

Fortunately, the open-source world doesn’t have to rely on the roadmaps created by networking vendors’ product managers; if there’s a big enough pain, someone will solve it.

Yesterday, I mentioned that a Cisco router running pre-standard IS-IS 3-way handshake (this is why you need it) interoperates with multiple implementations of RFC 5303. How’s that possible, and does it matter whether you configure the ancient Cisco routers (release 15.x) to use IETF 3-way handshake instead of the “proprietary” one?

I took a trip to the Wireshark land to figure out the details (you can download the capture file):

Dan Partelly figured out that we have to configure the standard (IETF) 3-way IS-IS handshake on old IOSv images. On the other hand, all IS-IS integration tests pass for IOSv and IOSvL2. I wondered what was going on.

Fortunately, a few months ago, I spent some time installing the client-side Edgeshark components on my laptop. All I needed to do was enable the edgeshark tool in my lab topology and restart the lab.