The early October 2021 Facebook outage generated a predictable phenomenon – couch epidemiologists became experts in little-known Bridging the Gap Protocol (BGP), including its Introvert and Extrovert variants. Unfortunately, I also witnessed several unexpected trips to Mount Stupid by people who should have known better.

To set the record straight: everyone’s been there, and the more vocal you tend to be on social media (including mailing lists), the more probable it is that you’ll take a wrong turn and end there. What matters is how gracefully you descend and what you’ve learned on the way back.

When I started creating the How Networks Really Work series, I wondered whether our subscribers (mostly seasoned networking engineers) would find it useful. Turns out at least some of them do; this is what a long-time subscriber sent me:

How Networks Really Work is great, it’s like looking from a plane and seeing how all the roads are connected to each other. I know networking just enough to design and manage a corporate network, but there are many things I have learned, used and forgotten along the way.

So, getting a broad vision helps me remember why I chose something and maybe solve my bad choices. There are many things that I may never use, but with the movement of all things in the cloud it’s great to know, or at least understand, how things really work.

Two weeks ago I replied to a battle-scar reaction to 7-layer OSI model, this time I’ll address a much more nuanced view from Russ White. Please read his article first (as always, it’s well worth reading) and when you come back we’ll focus on this claim:

The OSI Model does not accurately describe networks.

Like with any tool in your toolbox, you can view the 7-layer OSI model in a number of ways. In the case of OSI model, it can be used:

I should have known better, but I got pulled into another stretched VLANs for disaster recovery tweetfest. Surprisingly, most of the tweets were along the lines of you really shouldn’t be doing that and that would never work well, but then I guess I was only exposed to a small curated bubble of common sense… until this gem appeared in my timeline:

Interestingly, that’s exactly how IP works:

One of my readers sent me this interesting question:

It begs the question in how far graduated students with a degree in computer science or applied IT infrastructure courses (on university or college level or equivalent) are actually aware of networking fundamentals. I work for a vendor independent networking firm and a lot of my new colleagues are college graduates. Positively, they are very well versed in automation, scripting and other programming skills, but I never asked them what actually happens when a packet traverses a network. I wonder what the result would be…

I can tell you what the result would be in my days: blank stares and confusion. I “enjoyed” a half-year course in computer networking that focused exclusively on history of networking and academic view of layering, and whatever I know about networking I learned after finishing my studies.

I was speaking with a participant of an SDN event in Zurich after the presentations, and he made an interesting comment: whenever he experienced serious troubleshooting problems in his career, it was due to lack of understanding of networking fundamentals.

Let me give you a few examples: Do you know how ARP works? What is proxy ARP? How does TCP offload work and why is it useful? What is an Ethernet collision and when would you see one? Why do we need MLD in IPv6 neighbor discovery?

Whenever I’m ranting about vendors changing their data models or APIs with every other release, there is inevitably a vendor engineer chiming in, saying, “Life would be so much better if the customers wouldn’t insist on doing screen scraping for the last 50 years.”

While some of that screen scraping is pure inertia, we sometimes have good reasons to do it rather than use protocols like NETCONF, gNMI, or protobufs. In Episode 205 of Software Gone Wild, I’m discussing some of those reasons and exploring the gap between vendor theory and reality with Dinesh Dutt, who is unlucky enough to have become the world’s foremost expert on crappy network telemetry.

When I wrote about the anycast-ECMP-in-MPLS behavior in 2011, I had to use Cisco IOS to prove that ECMP worked, since Arista cEOS (running the Linux kernel for IP forwarding) didn’t install more than one equal-cost path into the Linux forwarding table.

Arista cEOS got better in the meantime; IPv4 ECMP works like a charm on cEOS release 4.35.02F. With the same lab topology I’d used in 2021, I was able to see the traffic spread across multiple nodes:

Here’s an interesting tidbit from the what took them so long department: Cloudflare One Client continuously measures end-to-end MTU and adjusts the local tunnel interface MTU size accordingly (warning: there’s a fair amount of dubious handwaving over the interesting details), generating ICMP packet-too-big messages as close to the source as possible.

I managed to avoid VPN clients most of my life, so I have no idea whether this is a “finally someone figured that out 🎉” moment or a late catch-up to what other VPN clients have been doing for ages. Feedback (in comments or otherwise) would be most welcome!

netlab release 26.03 is out. Here are the highlights:

• New Cisco IOS XR features: SR-MPLS, MPLS/VPN, MPLS 6PE, EVPN over MPLS, VRRP, BGP session parameters, routing policies, prefix filters, AS-path filters, community filters, and static routes
• Juniper cRPD support (mostly by @leec-666)
• EVPN/VXLAN on Juniper vPTX and vJunos-router by @rickycraft
• OpenBSD BGP support by @remilocherer
• SR-MPLS support for OSPFv2 on EOS, FRR, IOS XE, IOS XR
• EVPN/MPLS improvements, including using SR-MPLS transport

For even more details, check the release notes.

We haven’t implemented support for Cisco SD-WAN in netlab yet, and we might never do so; after all, netlab isn’t meant to be a kitchen sink of vendor-specific features. However, having an open-source tool that uses input and output files with standardized encoding (JSON or YAML) makes it easy to develop an independent solution that adds functionality.

That’s exactly what Sebastien d’Argoeuves did: he developed a solution that automates Cisco SD-WAN deployment after the corresponding netlab lab is started, and published it in a GitHub repo. If you’re an SD-WAN fan, you must give it a try ;)

Yesterday, I had a short presentation on netlab use cases during the NetBCN event. It covered a dozen examples, from rapid prototyping to testing network automation software and arguing with vendor TAC. I added the “use cases” part of the presentation to the standard netlab presentation; you can view the results on ipSpace.net (no account or registration required).

I decided it was high time to create EVPN/MPLS netlab integration tests and wanted to use the same approach I used for the EVPN/VXLAN ones:

• One of the PE-devices is the device we want to test
• The other PE-device is a device that is known to work (ideally, an FRRouting container).
• Bonus points if the other PE-device can generate operational data in JSON format. Using a device for which we already have a validation plugin is close to perfection.
• Add a P-router in the middle because MPLS.
• Attach some hosts to the two PE-devices (we’re testing two MAC-VRFs in the final version of the test)
• After validating everything that can reasonably be validated (OSPF session, IBGP session, EVPN AF on IBGP session), do the end-to-end pings and hope for the best.

This is the graph netlab created from the lab topology:

Bruce Davie published a nice article explaining why it makes little sense to use an algorithm that’s supposedly faster than Dijkstra’s in link-state routing protocols.

Other interesting data points from the article (and linked presentations):

• People are running (a few) thousands of routers in a single area
• Running Dijkstra’s algorithm on an emulated network with 2000 nodes took 100 msec… in 2003 (page 18 of this NANOG presentation).

It turns out (as I expected) that all the noise about the need for new routing protocols we were experiencing a few years ago was either due to bad implementations or coming from nerds looking for new toys to play with.

Someone recently asked me whether it’s possible to use netlab to build an MPLS/VPN (technically, BGP/MPLS IP VPN) lab with SR-MPLS core. Of course, let’s build a simple lab using Arista EOS and Linux containers to implement this topology:

Here’s the lab topology we’ll use (also available on GitHub):