Category: worth reading

Did you ever experience an out-of-the-blue BGP session flap after you were running that peering for months? As Dmytro Shypovalov explains in his latest blog post, it’s always MTU (just kidding, of course it’s always DNS, but MTU blackholes nonetheless result in some crazy behavior).

A long while ago I wrote a blog post along the lines of “it’s ridiculous to allow developers to deploy directly to a public cloud while burdening them with all sorts of crazy barriers when deploying to an on-premises infrastructure,” effectively arguing for self-service approach to on-premises deployments.

Not surprisingly, the reality is grimmer than I expected (I’m appalled at how optimistic my predictions are even though I always come across as a die-hard grumpy pessimist), as explained in The Shared Irresponsibility Model in the Cloud by Dan Hubbard.

For more technical details, watch cloud-focused ipSpace.net webinars, in particular the Cloud Security one.

My friend Marjan Bradeško wrote a great article describing how we tend to forget common sense and rely too much on technology. I would strongly recommend you read it and start thinking about the choices you make when building a network with magic software-intent-defined-intelligent technology from your preferred vendor.

I was telling you there’s no need to become a programmer over six years ago, but of course nobody ever listens to grumpy old engineers… which didn’t stop Ethan Banks from writing another excellent advice on the same theme: Don’t Become A Developer, But Use Their Tools.

We all knew it for a long time, now it’s finally official: IP fragmentation is broken, or as the ever-so-diplomatic IETF likes to call it, IP Fragmentation is Considered Fragile.

Justin Pietsch is back with another must-read article, this time focused on high-speed Ethernet switching ASICs. I’ve rarely seen so many adjacent topics covered in a single easy-to-read article.

In one of his recent blog posts Tom Hollingsworth described what I semi-consciously felt about the CCIE lab exam for at least 25 years: it’s full of contrived scenarios that look more like Iron Chef than real life.

I understand they had to make the lab harder and harder to stop cheating (because talking with candidates and flunking the incompetents is obviously not an option), and there’s only so much one can do with a limited set of technologies… but forcing networking engineers to find ever-more-devious ways to solve overly-complex problems is nothing else but fuel for rampant MacGyverism.

Anyway, I don’t think this mess will ever be fixed, so the only thing we can do is to enjoy the rant.

One of the weekend reads collected by Russ White contained a pointer to a hilarious description of blockchain - a solution in search of a problem. Here are a few quotes to get you started (and I had a really hard time selecting just a few):

I’ve never seen so much bloated bombast fall so flat on closer inspection.

At its core, blockchain is a glorified spreadsheet.

The only thing is that there’s a huge gap between promise and reality. It seems that blockchain sounds best in a PowerPoint slide.

Someone should use that article as a framework and replace blockchain with OpenFlow or SDN ;)

Years ago I was naive enough to participate in writing an IETF document. Three years later we finally managed to turn it into an RFC, and I decided that was enough for one lifetime.

But wait, it gets worse… as Geoff Huston argues in his article, the lengthy review process doesn’t necessarily result in better (or more precise) documents.

Seems like we managed to turn IETF into yet another standard body like IEEE, ISO or ITU/T.

Justin Pietsch published another must-read article, this time dealing with operational complexity of load balancers and IP multicast. Here are just a few choice quotes to get you started:

• A critical lesson I learned is that running out of capacity is the worst thing you can do in networking
• You can prevent a lot of problems if you can deep dive into an architecture and understand it’s tradeoffs and limitations
• Magic infrastructure is often extremely hard to troubleshoot and debug

You might find what he learned useful the next time you’re facing a unicorn-colored slide deck from your favorite software-defined or intent-based vendor ;))

Greg Ferro is back with some great technical content, this time explaining why hardware-based packet capture might return unexpected results.

Justin Pietsch published a fantastic recap of his experience running OSPF in AWS infrastructure. You MUST read what he wrote, here’s the TL&DR summary:

• Contrary to popular myths, OSPF works well on very large leaf-and-spine networks.
• OSPF nuances are really hard to grasp intuitively, and the only way to know what will happen is to run tests with the same codebase you plan to use in a production environment.

Dinesh Dutt made similar claims on one of our podcasts, and I wrote numerous blog posts on the same topic. Not that anyone would care or listen; it’s so much better to watch vendor slide decks full of the latest unicorn dust… but in the end, it’s usually not the protocol that’s broken, but the network design.

Brett Lykins published an excellent description of what an automation Minimum Viable Product could be.

Not surprisingly, he’s almost perfectly in sync with what we’ve been telling networking engineers in ipSpace.net Network Automation online course:

• Start small
• Go for quick wins
• Do read-only stuff before modifying device configurations
• Test, test, test…

Here’s an interesting factoid: when using default IS-IS configuration (running L1 + L2 on all routers in your network), every router inserts every IP prefix from anywhere in your network into L2 topology… at least on Junos.

For more details read this article by Chris Parker. I also wrote about that same problem in 2011.

When someone sent me a presentation on seamless MPLS a long while ago my head (almost) exploded just by looking at the diagrams… or in the immortal words of @amyengineer:

“If it requires a very solid CCIE on an obscure protocol mix at 4am, it is a bad design” - Peter Welcher, genius crafter of networks, granter of sage advice.

Turns out I was not that far off… Dmytro Shypovalov documented the underlying complexity and a few things that can go wrong in Seamless Suffering.