The early October 2021 Facebook outage generated a predictable phenomenon – couch epidemiologists became experts in little-known Bridging the Gap Protocol (BGP), including its Introvert and Extrovert variants. Unfortunately, I also witnessed several unexpected trips to Mount Stupid by people who should have known better.

To set the record straight: everyone’s been there, and the more vocal you tend to be on social media (including mailing lists), the more probable it is that you’ll take a wrong turn and end there. What matters is how gracefully you descend and what you’ve learned on the way back.

When I started creating the How Networks Really Work series, I wondered whether our subscribers (mostly seasoned networking engineers) would find it useful. Turns out at least some of them do; this is what a long-time subscriber sent me:

How Networks Really Work is great, it’s like looking from a plane and seeing how all the roads are connected to each other. I know networking just enough to design and manage a corporate network, but there are many things I have learned, used and forgotten along the way.

So, getting a broad vision helps me remember why I chose something and maybe solve my bad choices. There are many things that I may never use, but with the movement of all things in the cloud it’s great to know, or at least understand, how things really work.

Two weeks ago I replied to a battle-scar reaction to 7-layer OSI model, this time I’ll address a much more nuanced view from Russ White. Please read his article first (as always, it’s well worth reading) and when you come back we’ll focus on this claim:

The OSI Model does not accurately describe networks.

Like with any tool in your toolbox, you can view the 7-layer OSI model in a number of ways. In the case of OSI model, it can be used:

I should have known better, but I got pulled into another stretched VLANs for disaster recovery tweetfest. Surprisingly, most of the tweets were along the lines of you really shouldn’t be doing that and that would never work well, but then I guess I was only exposed to a small curated bubble of common sense… until this gem appeared in my timeline:

Interestingly, that’s exactly how IP works:

One of my readers sent me this interesting question:

It begs the question in how far graduated students with a degree in computer science or applied IT infrastructure courses (on university or college level or equivalent) are actually aware of networking fundamentals. I work for a vendor independent networking firm and a lot of my new colleagues are college graduates. Positively, they are very well versed in automation, scripting and other programming skills, but I never asked them what actually happens when a packet traverses a network. I wonder what the result would be…

I can tell you what the result would be in my days: blank stares and confusion. I “enjoyed” a half-year course in computer networking that focused exclusively on history of networking and academic view of layering, and whatever I know about networking I learned after finishing my studies.

I was speaking with a participant of an SDN event in Zurich after the presentations, and he made an interesting comment: whenever he experienced serious troubleshooting problems in his career, it was due to lack of understanding of networking fundamentals.

Let me give you a few examples: Do you know how ARP works? What is proxy ARP? How does TCP offload work and why is it useful? What is an Ethernet collision and when would you see one? Why do we need MLD in IPv6 neighbor discovery?

Imagine you decide to believe the marketing story of your preferred networking vendor and start using the REST API to configure their devices. That probably involves some investment in automation or orchestration tools, as nobody in their right mind wants to use curl or Postman to configure network devices.

A few months later, after your toolchain has been thoroughly tested, you decide to upgrade the operating system on the network devices, and everything breaks. The root cause: the vendor changed their API or the data model between software releases.

I wanted to test a loop prevention when propagating summary LSA across areas scenario (more about that in another blog post) using the lab topology I developed for the When OSPF Becomes a Distance Vector Protocol article.

I started the lab with the FRRouting routers and configured OSPF area ranges. Astonishingly, I discovered that the more-specific prefixes from an area appear as summary routes in the backbone area even when the area range is configured. When I tried to reproduce the scenario a few days later, it turned out to be a timing quirk (I didn’t wait long enough), but my squirrelly mind was already investigating.

After discussing networking layers and addressing, it’s time to focus on moving packets across a network. Vendors love to use ill-defined terms like switching instead of forwarding, routing, or bridging, so let’s start with the terminology.

Connecting all relevant devices to a single cable would indubitably simplify any networking stack, but unfortunately, we’re almost never that lucky. We need devices in the network (typically with multiple interfaces) that perform packet forwarding between end nodes.

Gabriel sent me a pointer to a blog post by Rudolph Bott describing the details of BGP Unnumbered implementations on Nokia, Juniper, and Bird.

Even more interestingly, Rudolph points out the elephant I completely missed: RFC 8950 refers to RFC 2545, which requires a GUA IPv6 next hop in BGP updates (well, it uses the SHALL wording, which usually means “troubles ahead”). What do you do if you’re running EBGP on an interface with no global IPv6 addresses? As expected, vendors do different things, resulting in another fun interoperability exercise.

Finally, there’s RFC 7404 that advocates LLA-only infrastructure links, so we might find the answer there. Nope; it doesn’t even acknowledge the problem in the Caveats section.

For even more information, read the Unnumbered IPv4 Interfaces and BGP in Data Center Fabrics blog posts.

The amount of weird stuff we discover in netlab integration tests is astounding, or maybe I have a knack for looking into the wrong dark corners (my wife would definitely agree with that). Today’s special: when having two next hops kills a static route.

TL&DR: default ARP settings on a multi-subnet Linux host are less than optimal.

We use these principles when creating netlab integration tests:

• They should contain a single device-under-test and a bunch of attached probes.
• They should test a single feature.
• They should not rely on the device-under-test. All validation has to be done on probes.

How do you test static routes under these restrictions? Here’s what we did:

I apologize for the rant; I have to vent my frustration with people whose quantity of opinions seems to be exceeding their experience (or maybe they’re coming from an alternate universe with different laws of physics, which would be way cool but also unlikely). You’ve been warned; please feel free to move on or skip the rant part of the blog post.

Rant mode: ON

This is the (unedited) gem I received after making some of my EVPN videos public:

When I was updating the Network Migration with BGP Local-AS Feature blog post, I wanted to execute the same command (show ip bgp) on all routers in my network.

Not a problem: since Dan Partelly added the netlab exec command, it’s as simple as netlab exec * show ip bgp. Well, not exactly; there are still a few quirks.

Daryll Swer posted a very informative response to my NAT Traversal Mess blog post, focusing on:

Punching holes through that firewall is equivalent to establishing NAT translations.

It would be a shame to let that response wither as small print at the bottom of a blog post; here it is:

As I started describing the iBGP Local-AS Details, I discovered that anything I ever wrote about the BGP local-as feature was hosted on a now-extinct website. Fortunately, I found a copy of that article in the Internet Archive and migrated it to the ipSpace.net blog. Hope you’ll like it ;)