Anne Baretta found a great video describing the October 2018 GitHub failure. Here’s the TL&DW:

• The failure was caused by a short (~ 1 minute) disconnect of the primary data center
• The database replicas failed over to the secondary data center, but that failover was never tested and of course some stuff didn’t work.
• In the meantime, batch jobs modified data in the primary data center, making the two replicas out-of-sync.
• It took them over 24 hours to clean up the mess.

One of my readers asked for my opinion about the following masterpiece posted on (where else) LinkedIn¹:

We added just a few small features in netlab release 1.6.1¹:

• Markdown reports: netlab report command can produce Markdown-formatted reports, making it extremely easy to include them in your documentation (assuming you’re using Markdown to write it)
• If you’re using BGP in your labs, you can generate reports on BGP autonomous systems and BGP neighbors.
• I made locations of default files configurable. I’m using this feature in large projects where I want to have a shared set of project-wide defaults for topologies stored in different directories.

More than a dozen years after the SDN brouhaha erupted, some people still haven’t got the memo on the obsolescence of CLI. For example, Julia Evans tries to make people comfortable with the command line. Has nobody told her it’s like teaching COBOL?

On a more serious note: you OUGHT TO master Linux CLI and be comfortable using CLI commands on network devices and servers. Her article has tons of useful tips and is definitely worth reading.

I’m publishing a link to a free ipSpace.net video several times each month, usually with a notice saying you need free subscription to watch the video. I had to put that limitation in place when I was hosting videos on AWS S3 – unlimited streaming could explode my AWS bill.

Recently I moved the video storage to Cloudflare R2. Cloudflare claims they will never charge egress fees, and as long as that’s true (and they don’t start chasing me for generating too much traffic) I see no reason to bother you with registration and login procedures – starting immediately, you can watch the free ipSpace.net videos without an ipSpace.net account.

Sharada Yeluri (Senior Director of Engineering at Juniper Networks) wrote a long article describing the connectivity requirements of AI workloads and new approaches to Ethernet fabrics. Definitely worth reading if you’re interested in these topics.

Long story short: I decided to create open-source BGP configuration labs, and (so far) created a superset of labs we used in an ancient Advanced BGP Configuration and Troubleshooting (ABCT) course

netlab release 1.6.0 has (probably) the longest release notes so far as it contains so many user-visible new features including:

New Commands

Some users were complaining how complex it was to use netlab create command to create graphs, inspect data structures, or create custom reports. They might find the new commands easier to use:

Wouldn’t it be nice if your home router (CPE) could use DSL (or slow-speed fibre) and LTE connection at the same time? Even better: run a single TCP session over both links? The answer to both questions is YES, of course it could do that, if only your service provider would be interested in giving you that option.

We solved similar problems with multilink PPP in the networking antiquity, today you could use a CPE with an MP-TCP proxy combined with a Hybrid Access Gateway in the service provider network. For more details, read the excellent Increasing broadband reach with Hybrid Access Networks article by prof. Olivier Bonaventure and his team.

Gerben Wierda published a nice description of common reactions to new unicorn-dust-based technologies:

• Eyes that glaze over
• Eyes like saucers
• Eyes that narrow

He uses generative AI as an example to explain why it might be a bad idea that people in the first two categories make strategic decisions, but of course nothing ever stops people desperately believing in vendor fairy tales, including long-distance vMotion, SDN or intent-based networking.

Brian Carpenter published a list of Multipath TCP resources to one of the IETF mailing lists¹:

• Modern Multipath Transport Protocols – an ebook by prof. Olivier Bonaventure describing QUIC, multipath TCP and multipath QUIC.
• Multipath TCP Wiki
• Multipath TCP for Linux
• Multipath TCP Python extension module

You might also want to listen to the Multipath TCP podcast we recorded with Apple engineers in 2019.

Smart engineers were forever using Linux (in particular, its traffic control/queue discipline functionality) to simulate WAN link impairment. Unfortunately, there’s a tiny hurdle you have to jump across: the tc CLI is even worse than iptables.

A long while ago someone published a tc wrapper that simulates shitty network connections and (for whatever reason) decided to call it Comcast. It probably does the job, but I would prefer to have something in Python. Daniel Dib found just that – tcconfig – and used it to simulate WAN link behavior on VMware vSphere.

Bruce Davie collected numerous articles describing various aspects of early Internet history and pre-Internet days, including A Brief History of the Internet and The Design Philosophy of the DARPA Internet Protocols.

Have fun ;)

Justus sent me an email with an interesting link:

Since you love to make comparisons to the good ol’ thick yellow cable while I as a mid-30 year old adult have no idea what you are talking about: Computerphile made a video about Ethernet on the occasion of its 50th birthday. The university of Nottingham got the chance to show their museum pieces :-) (about 8:45 min).

Thanks a million!

Emile Aben is describing an interesting behavior observed in the Wild West of the global Internet: someone started announcing BGP paths with an unknown attribute, which (regardless of RFC 7606) triggered some BGP session resets.

One would have hoped we learned something from the August 2010 incident (supposedly caused by a friend of mine 😜), but it looks like some things never change. For more details, watch the Network Security Fallacies and Internet Routing Security webinar.