Category: worth reading

What happens when you let a bunch of people work on different aspects of a solution without them ever talking to each other? You get DNS over IPv6. As nicely explained by Geoff Huston, this is just one of the bad things that could happen:

Around 30 years after we got the first website, the powers that be realized it might make sense to put this is how you access a web server information (including its IPv4 and IPv6 address, and HTTP(S) support information) directly into DNS, using HTTPS Resource Records. It took us long enough 🤷‍♂️

George Davitiani put together a lovely proof-of-concept using GitHub actions to deploy modified configurations to network devices. Even better, he documented the whole setup, and the way to reproduce it. I’m positive you’ll find a few ideas browsing through what he did.

Daniel Teycheney decided not to renew his CCNP status and used this opportunity to publish his thoughts on IT certifications. Not surprisingly, I agree with most of the things he said, but I never put it in writing so succinctly.

Red Pill Warning: Reading his blog post might damage your rosy view of the networking industry. You’ve been warned ;)

Cloudflare experienced a significant outage in early November 2023 and published a detailed post-mortem report. You should read the whole report; here are my CliffsNotes:

• Regardless of how much redundancy you have, sometimes all systems will fail at once. Having redundant systems decreases the probability of total failure but does not reduce it to zero.
• As your systems grow, they gather hidden- and circular dependencies.
• You won’t uncover those dependencies unless you run a full-blown disaster recovery test (not a fake one)
• If you don’t test your disaster recovery plan, it probably won’t work when needed.

Also (unrelated to Cloudflare outage):

Julia Evans wrote another must-read article (if you’re using Git): git rebase: what can go wrong?

I often use git rebase to clean up the commit history of a branch I want to merge into a main branch or to prepare a feature branch for a pull request. I don’t want to run it unattended – I’m always using the interactive option – but even then, I might get into tight spots where I can only hope the results will turn out to be what I expect them to be. Always have a backup – be it another branch or a copy of the branch you’re working on in a remote repository.

Julia Evans wrote another great article explaining confusing git terminology. Definitely worth reading if you want to move past simple recipes or reminiscing about old days.

Almost exactly a decade ago I wrote about a paper describing how IBGP migrations can cause forwarding loops and how one could reorder BGP reconfiguration steps to avoid them.

One of the paper’s authors was Laurent Vanbever who moved to ETH Zurich in the meantime where his group keeps producing great work, including the Chameleon tool (code on GitHub) that can tame transient loops while reconfiguring BGP. Definitely something worth looking at if you’re running a large BGP network.

My good friend Tiziano Tofoni finally created an English version of his evergreen classic BGP from theory to practice with co-authors Antonio Prado and Flavio Luciani.

I had the Italian version of the book since the days I was running SDN workshops with Tiziano in Rome, and it’s really nice to see they finally decided to address a wider market.

Also, you know what would go well with that book? Free open-source BGP configuration labs of course 😉

I always tell networking engineers who aspire to be more than VLAN-munging CLI jockeys to get fluent with Git. I should also be telling them that while doing local version control is the right thing to do, you should always have backups (in this case, a remote repository).

I’m eating my own dog food¹ – I’m using a half dozen Git repositories in ipSpace.net production². If they break, my blog stops working, and I cannot publish new documents³.

Now for a fun fact: Git is not transactionally consistent.

Gary Marcus wrote an interesting essay describing the failure of self-driving cars to face the unknown unknowns. The following gem from his conclusions applies to AI in general:

In a different world, less driven by money, and more by a desire to build AI that we could trust, we might pause and ask a very specific question: have we discovered the right technology to address edge cases that pervade our messy really world? And if we haven’t, shouldn’t we stop hammering a square peg into a round hole, and shift our focus towards developing new methodologies for coping with the endless array of edge cases?

Obviously that’s not going to happen, we’ll keep throwing more GPU power at the problem trying to solve it by brute force.

Talking about BGP routing policy mechanisms is nice, but it’s even better to see how real Internet Service Providers use those tools to implement real-life BGP routing policy.

Getting that information is incredibly hard as everyone considers their setup a secret sauce. Fortunately, there are a few exceptions; Pim van Pelt described the BGP Routing Policy of IPng Networks in great details. The article is even more interesting as he’s using Bird2 configuration language that looks almost like a programming language (as compared to the ancient route-maps used by vendors focused on “industry-standard” CLI).

Have fun!

More than thirteen years after I started creating vendor-neutral webinars, it’s time for another change¹: the ipSpace.net subscriptions became perpetual. If you have an active ipSpace.net subscription, it will stay valid indefinitely² (and I’ll stop nagging you with renewal notices).

Wow, Free Lunch?

Sadly, that’s not the case.

Bruce Davie published an interesting overview article about Large Language Models. It would be worth reading just for the copious links to in-depth article; I particularly like his conclusions:

We mistake performance (producing realistic text) for competence (understanding the world).

Having a model for language is different from having a model of the world.

And that’s a perfect explanation why it makes no sense to expect ChatGPT and friends to produce picture-perfect device configurations or always-working code.

Anne Baretta found a great video describing the October 2018 GitHub failure. Here’s the TL&DW:

• The failure was caused by a short (~ 1 minute) disconnect of the primary data center
• The database replicas failed over to the secondary data center, but that failover was never tested and of course some stuff didn’t work.
• In the meantime, batch jobs modified data in the primary data center, making the two replicas out-of-sync.
• It took them over 24 hours to clean up the mess.