Blog Posts in November 2019
Worth Reading: Resilience Engineering
Starting with my faking disaster recovery tests blog post Terry Slattery wrote a great article delving into the intricacies of DR testing, types of expected disasters, and resilience engineering. As always, a fantastic read from Terry.
IP Fabric with Gian-Paolo Boarina on Software Gone Wild
No, we were not talking about IP fabrics in general - IP Fabric is a network management software (oops, network assurance platform) Gian Paolo discovered a while ago and thoroughly tested in the meantime.
He was kind enough to share what he found in Episode 107 of Software Gone Wild, and as Chris Young succinctly summarized: “it’s really sad what we still get excited about something 30 years after it was first promised”… but maybe this time it really works ;)
Upcoming Events and Webinars (December 2019)
The registration is still open for the Using VXLAN to Build Active-Active Data Centers workshop on December 3rd, but if you can’t make it to Zurich you might enjoy these live sessions we’ll run in December 2019:
- In the last session of autumn 2019 network automation course (December 5th) we’ll have Jeremy Schulman talking about ChatOps and Slack integration;
- On December 10th Matthias Luft will continue the Cloud Security series with a session focused on identity and access management (IAM);
- Remember the Business Aspects of Networking Technologies idea I started in April? Finally I found time for the second session (on December 12th), this time focused on addressing the business challenges instead of getting excited about technology;
- We’ll conclude the Autumn 2019 webinar season with another session in the EVPN Deep Dive saga on December 17th. This time we’ll have Krzysztof Szarkowicz, the author of MPLS in the SDN Era talk about MPLS-based EVPN.
All webinars I mentioned above are accessible with Standard ipSpace.net Subscription, and you’ll need Expert Subscription to enjoy the automation course contents.
Is There a Future for Networking Engineers?
Someone sent me this observation after reading my You Cannot Have Public Cloud without Networking blog post:
As much as I sympathize with your view, scales matter. And if you make ATMs that deal with all the massive client population, the number of bank tellers needed will go down. A lot.
Based on what I read a while ago a really interesting thing happened in financial industry: while the number of tellers went down, number of front-end bank employees did not go down nearly as dramatically, they just turned into “consultants”.
The EVPN Dilemma
I got an interesting set of questions from a networking engineer who got stuck with the infamous “let’s push the **** down the stack” challenge:
So, I am a rather green network engineer trying to solve the typical layer two stretch problem.
I could start the usual “friends don’t let friends stretch layer-2” or “your business doesn’t need that” windmill fight, but let’s focus on how the vendors are trying to sell him the “perfect” solution:
Worth Reading: Early History of Usenet
Steve Bellovin wrote a great series of articles describing the early history of Usenet. The most interesting part in the “security and authentication” part was probably this gem:
That left us with no good choices. The infrastructure for a cryptographic solution was lacking. The uux command rendered illusory any attempts at security via the Usenet programs themselves. We chose to do nothing. That is, we did not implement fake security that would give people the illusion of protection but not the reality.
A lot of other early implementers chose the same route, resulting in SMTP, BGP… which wouldn’t be a problem if someone kept track of that and implemented security a few years later. Unfortunately we considered those problems solved and moved on to chase other squirrels. We’re still paying the interest on that technical debt.
Worth Reading: Going Back to Being an Engineer
Another great advice from Charity Majors: does it make sense to go back to being an engineer after being a manager for a few years?
Personal note: finding a great replacement for my CTO role was probably the best professional decision I ever made ;)
Video: Breaking the End-to-End Principle
Original TCP/IP and OSI network stacks had relatively clean layered architecture (forgetting the battle scars for the moment) and relied on end-to-end principle to keep the network core simple.
As always, no good deed goes unpunished - “creative” individuals trying to force-fit their misdesigned star-shaped pegs into round holes, and networking vendors looking for competitive advantage quickly destroyed the idea with tons of middlebox devices, ranging from firewalls and load balancers to NAT, WAN optimization, and DPI monstrosities.
Guest Speakers in the Networking in Public Cloud Deployments Online Course
We are proud to announce a great lineup of guest speakers for the first Networking in Public Cloud Deployments course that will run in Spring 2020:
- Joep Piscaer will dive into what changes public clouds bring and what these changes mean for you, as well as what developers and other consumers of cloud resources expect from you in the new public cloud, DevOps and Infrastructure-as-Code world.
- Ned Bellavance will review the principles of Infrastructure as Code (IaC) and how they apply to public cloud solutions. Then he will take a look at the landscape of IaC tools that exist and examine their pros and cons.
- Howard Marks will review the types of storage available across public clouds, how they differ between cloud providers and the applications and pitfalls associated with each of them.
- Connecting on-premises data centers or office locations to a public cloud has some unique challenges. Ed Horley will help you create a framework and a checklist to make sure you have the required redundancy, throughput, routing, and security all baked in from day one.
- Matthias Luft will cover the aspects of securing your public cloud deployments.
- Justin Warren will explain how to make good tradeoffs between resilient hardware and resilient software.
Sounds interesting? The first Networking in Public Cloud Deployments course will start on February 11th, 2020, but the minute you register you'll be able to start studying the materials (over 100 hours of content). There’s just one thing you have to do: click the Register button.
Automation Solution: L3VPN Topology
Here’s another “let’s use network automation tools to create reports we couldn’t get in the past” (like IP multicast trees) solution coming from an attendee in our network automation course: Paddy Kelly created L3VPN graphs detailing PE-to-CE connectivity using Cisco’s pyATS to parse the Cisco IOS printouts.
You’ll find dozens of other interesting solutions on our Sample Network Automation Solutions page - all of them were created by networking engineers who knew almost nothing about network automation or open-source automation tools when they started our automation course.
Using BGP for Firewall High Availability: Design and Software Upgrades
Remember the “BGP as High Availability Protocol” article Nicola Modena wrote a few months ago? He finally found time to extend it with BGP design considerations and a description of a seamless-and-safe firewall software upgrade procedure.
Facts and Fiction: BGP Is a Hot Mess
Every now and then a smart person decides to walk away from their competence zone, and start spreading pointless clickbait opinions like BGP is a hot mess.
Like any other technology, BGP is just a tool with its advantages and limitations. And like any other tool, BGP can be used sloppily… and that’s what’s causing the various problems and shenanigans everyone is talking about.
Just in case you might be interested in facts instead of easy-to-digest fiction:
Worth Reading: the Lure of the Easy Button
Russ White wrote a great blog post explaining why you have to understand the problem you’re solving instead of blindly believing the $vendor slide deck… or as I said a long time ago, think about how you’ll troubleshoot your network in because you won’t be able to reformat it once it crashes.
Your First Public Cloud Deployment Should Be Small
I’ve seen successful public (infrastructure) cloud deployments… but also spectacular failures. The difference between the two usually comes down to whether the team deploying into a public cloud environment realizes they’re dealing with an unfamiliar environment and acts accordingly.
Please note that I’m not talking about organizations migrating their email to Office 365. While that counts as public cloud deployment when an industry analyst tries to paint a rosy picture of public cloud acceptance, I’m more interested in organizations using compute, storage, security and networking public cloud infrastructure.
Can We Really Use Millions of VXLAN Segments?
One of my readers sent me a question along these lines…
VXLAN Network Identifier is 24 bit long, giving 16 us million separate segments. However, we have to map VNI into VLANs on most switches. How can we scale up to 16 million segments when we have run out of VLAN IDs? Can we create a separate VTEP on the same switch?
VXLAN is just an encapsulation format and does not imply any particular switch architecture. What really matters in this particular case is the implementation of the MAC forwarding table in switching ASIC.
Stretched VLANs and Failing Firewall Clusters
After publishing the Disaster Recovery Faking, Take Two blog post (you might want to read that one before proceeding) I was severely reprimanded by several people with ties to virtualization vendors for blaming virtualization consultants when it was obvious the firewall clusters stretched across two data centers caused the total data center meltdown.
Let’s chase that elephant out of the room first. When you drive too fast on an icy road and crash into a tree who do you blame?
- The person who told you it’s perfectly OK to do so;
- The tire manufacturer who advertised how safe their tires were?
- The tires for failing to ignore the laws of physics;
- Yourself for listening to bad advice
For whatever reason some people love to blame the tires ;)
Stretched Layer-2 Subnets in Azure
Last Thursday morning I found this gem in my Twitter feed (courtesy of Stefan de Kooter)
Greg Cusanza in #BRK3192 just announced #Azure Extended Network, for stretching Layer 2 subnets into Azure!
As I know a little bit about how networking works within Azure, and I’ve seen something very similar a few times in the past, I was able to figure out what’s really going on behind the scenes in a few seconds… and got reminded of an old Russian joke I found somewhere on Quora:
Video: Putting the Networking Layers Together
The previous videos from the How Networks Really Work webinar covered an overview of networking challenges and the importance of networking layers.
Now it’s time to put it all together.
Explore the Content Outline of Our Networking in Public Clouds Online Course
A few days ago we published the content outline for our Networking in Public Clouds online course.
We’ll start with the basics, explore the ways to automate cloud deployments (after all, you wouldn’t want to repeat the past mistakes and configure everything with a GUI, would you?), touch on compute and storage infrastructure, and the focus on the networking aspects of public cloud deployments including:
VMware NSX-T and Geneve Q&A
A Network Artist left a lengthy comment on my Brief History of VMware NSX blog post. He raised a number of interesting topics, so I decided to write my replies as a separate blog post.
Using Geneve is an interesting choice to be made and while the approach has it’s own Pros and Cons, I would like to stick to VXLAN if I were to recommend to someone for few good reasons.
The main reason I see for NSX-T using Geneve instead of VXLAN is the need for additional header fields to carry metadata around, and to implement Network Services Header (NSH) for east-west service insertion.
Executing a Jinja2 Loop for a Subset of Elements
Imagine you want to create a Jinja2 report that includes only a select subset of elements of a data structure… and want to have header, footer, and element count in that report.
Traditionally we’d solve that challenge with an extra variable, but as Jinja2 variables don’t survive loop termination, the code to do that in Jinja2 gets exceedingly convoluted.
Fortunately, Jinja2 provides a better way: using a conditional expression to select the elements you want to iterate over.
Maybe It's Time We Start Appreciating Standards
A friend of mine sent me a short message including…
There is a number of products that recently arrived or are coming to market using group encryption systems for IP networks, but are (understandably) not using IPsec.
… which triggered an old itch of mine caused by the “We don’t need no IETF standards, code is king” stupidity.
Why Are You Always so Negative?
During the last Tech Field Day Extra @ CLEUR, one of the fellow delegates asked me about my opinion on technology X (don’t remember the details, it was probably one of those over-hyped four-letter technologies). As usual, I started explaining the drawbacks, and he quickly stopped me with a totally unexpected question: “Why do you always tend to be so negative?”
That question has been haunting me for months… and here are a few potential answers I came up with.