How GitHub Learned How Hard Distributed Systems Are
Anne Baretta found a great video describing the October 2018 GitHub failure. Here’s the TL&DW:
- The failure was caused by a short (~ 1 minute) disconnect of the primary data center
- The database replicas failed over to the secondary data center, but that failover was never tested and of course some stuff didn’t work.
- In the meantime, batch jobs modified data in the primary data center, making the two replicas out-of-sync.
- It took them over 24 hours to clean up the mess.
Engagement Farming
One of my readers asked for my opinion about the following masterpiece posted on (where else) LinkedIn1:
netlab 1.6.1: BGP Reports, Markdown Support
We added just a few small features in netlab release 1.6.11:
- Markdown reports: netlab report command can produce Markdown-formatted reports, making it extremely easy to include them in your documentation (assuming you’re using Markdown to write it)
- If you’re using BGP in your labs, you can generate reports on BGP autonomous systems and BGP neighbors.
- I made locations of default files configurable. I’m using this feature in large projects where I want to have a shared set of project-wide defaults for topologies stored in different directories.
Getting Comfortable with the Command Line
More than a dozen years after the SDN brouhaha erupted, some people still haven’t got the memo on the obsolescence of CLI. For example, Julia Evans tries to make people comfortable with the command line. Has nobody told her it’s like teaching COBOL?
On a more serious note: you OUGHT TO master Linux CLI and be comfortable using CLI commands on network devices and servers. Her article has tons of useful tips and is definitely worth reading.
Free Subscription No Longer Needed to Watch the ipSpace.net Videos
I’m publishing a link to a free ipSpace.net video several times each month, usually with a notice saying you need free subscription to watch the video. I had to put that limitation in place when I was hosting videos on AWS S3 – unlimited streaming could explode my AWS bill.
Recently I moved the video storage to Cloudflare R2. Cloudflare claims they will never charge egress fees, and as long as that’s true (and they don’t start chasing me for generating too much traffic) I see no reason to bother you with registration and login procedures – starting immediately, you can watch the free ipSpace.net videos without an ipSpace.net account.
Worth Reading: Networking for AI Workloads
Sharada Yeluri (Senior Director of Engineering at Juniper Networks) wrote a long article describing the connectivity requirements of AI workloads and new approaches to Ethernet fabrics. Definitely worth reading if you’re interested in these topics.
New Project: BGP Hands-On Labs
Long story short: I decided to create open-source BGP configuration labs, and (so far) created a superset of labs we used in an ancient Advanced BGP Configuration and Troubleshooting (ABCT) course
netlab 1.6.0: New Commands, Reports, and External Connectivity
netlab release 1.6.0 has (probably) the longest release notes so far as it contains so many user-visible new features including:
New Commands
Some users were complaining how complex it was to use netlab create command to create graphs, inspect data structures, or create custom reports. They might find the new commands easier to use:
Worth Reading: MP-TCP in Hybrid Access Networks
Wouldn’t it be nice if your home router (CPE) could use DSL (or slow-speed fibre) and LTE connection at the same time? Even better: run a single TCP session over both links? The answer to both questions is YES, of course it could do that, if only your service provider would be interested in giving you that option.
We solved similar problems with multilink PPP in the networking antiquity, today you could use a CPE with an MP-TCP proxy combined with a Hybrid Access Gateway in the service provider network. For more details, read the excellent Increasing broadband reach with Hybrid Access Networks article by prof. Olivier Bonaventure and his team.
Worth Reading: Eyes Like Saucers
Gerben Wierda published a nice description of common reactions to new unicorn-dust-based technologies:
- Eyes that glaze over
- Eyes like saucers
- Eyes that narrow
He uses generative AI as an example to explain why it might be a bad idea that people in the first two categories make strategic decisions, but of course nothing ever stops people desperately believing in vendor fairy tales, including long-distance vMotion, SDN or intent-based networking.