Blog Posts in October 2018
… updated on Friday, January 15, 2021 17:02 UTC
Internet Routing Security: It’s All About Business…
A few years ago I got cornered by an enthusiastic academic praising the beauties of his cryptography-based system that would (after replacing the whole Internet) solve all the supposed woes we’re facing with BGP today.
His ideas were technically sound, but probably won’t ever see widespread adoption – it doesn’t matter if you have great ideas if there’s not enough motivation to implementing them (The Myths of Innovation is a mandatory reading if you’re interested in these topics).
Observability Is the New Black
In early October I had a chat with Dinesh Dutt discussing the outline of the webinar he’ll do in November. A few days later Fastly published a blog post on almost exactly the same topic. Coincidence? Probably… but it does seem like observability is the next emerging buzzword, and Dinesh will try to put it into perspective answering these questions:
netdev 0x12 Update on Software Gone Wild
In recent years Linux networking started evolving at an amazing pace. You can hear about all the cool new stuff at netdev conference… or listen to Episode 94 of Software Gone Wild to get a CliffsNotes version.
Roopa Prabhu, Jamal Hadi Salim, and Tom Herbert joined Nick Buraglio and myself and we couldn’t help diverging into the beauties of tc, and the intricacies of low-latency forwarding before coming back on track and started discussing cool stuff like:
What’s the Big Deal with Validation?
This blog post was initially sent to subscribers of my mailing list. Subscribe here.
In his Intent-Based Networking Taxonomy blog post Saša Ratković mentioned real-time change validation as one of the requirements for a true intent-based networking product.
Old-time networking engineers would instinctively say “sure, we need that” while most everyone else might be totally flabbergasted. After all, when you create a VM, the VM is there (or you’d get an error message), and when you write to a file and sync the file system the data is stored, right?
As is often the case, networking is different.
VMware NSX: The Good, the Bad and the Ugly
After four live sessions we finished the VMware NSX Technical Deep Dive webinar yesterday. Still have to edit the materials, but right now the whole thing is already over 6 hours long, and there are two more guest speaker sessions to come.
Anyways, in the previous sessions we covered all the good parts of NSX and a few of the bad ones. Everything that was left for yesterday were the ugly parts.
Figuring Out AWS Networking
One of my friends reviewing the material of my AWS Networking webinar sent me this remark:
I'm always interested in hearing more about how AWS network works under the hood – it’s difficult to gain that knowledge.
As always, it’s almost impossible to find out the behind-the-scenes details, and whatever Amazon is telling you at their re:Invent conference should be taken with a truckload of salt… but it’s relatively easy to figure out a lot of things just by observing them and performing controlled experiments.
Automation Win: Configure Cisco ACI with an Ansible Playbook
This blog post was initially sent to subscribers of my mailing list. Subscribe here.
Following on his previous work with Cisco ACI Dirk Feldhaus decided to create an Ansible playbook that would create and configure a new tenant and provision a vSRX firewall for the tenant when working on the Create Network Services hands-on exercise in the Building Network Automation Solutions online course.
New: Expert ipSpace.net Subscription
Earlier this month I got this email from someone who had attended one of my online courses before and wanted to watch another one of them:
Is it possible for you to bundle a 1 year subscription at no extra cost if I purchase the Building Next-Generation Data Center course?
We were planning to do something along these lines for a long time, and his email was just what I needed to start a weekend-long hackathon.
End result: Expert ipSpace.net Subscription. It includes:
Leaf-and-Spine Fabric Myths (Part 3)
Evil CCIE concluded his long list of leaf-and-spine fabric myths (more in part 1 and part 2) with a layer-2 fabric myth:
Layer 2 Fabrics can't be extended beyond 2 Spine switches. I had a long argument with a $vendor guys on this. They don't even count SPB as Layer 2 fabric and so forth.
The root cause of this myth is the lack of understanding of what layer-2, layer-3, bridging and routing means. You might want to revisit a few of my very old blog posts before moving on: part 1, part 2, what is switching, layer-3 switches and routers.
MUST READ: Operational Security Considerations for IPv6 Networks
A team of IPv6 security experts I highly respect (including my good friends Enno Rey, Eric Vyncke and Merike Kaeo) put together a lengthy document describing security considerations for IPv6 networks. The document is a 35-page overview of things you should know about IPv6 security, listing over a hundred relevant RFCs and other references.
No wonder enterprise IPv6 adoption is so slow – we managed to make a total mess.
Event-Driven Network Automation in Network Automation Online Course
Event-driven automation (changing network state and/or configuration based on events) is the holy grail of network automation. Imagine being able to change routing policies (or QoS settings, or security rules) based on changes in the network.
We were able to automate simple responses with on-box solutions like Embedded Event Manager (EEM) available on Cisco IOS for years; modern network automation tools allow you to build robust solutions that identify significant events from the noise generated by syslog messages, SNMP traps and recently streaming telemetry, and trigger centralized responses that can change the behavior of the whole network.
Why Is Network Automation such a Hot Topic?
This blog post was initially sent to subscribers of my SDN and Network Automation mailing list. Subscribe here.
One of my readers asked a very valid question when reading the Why Is Network Automation So Hard blog post:
Why was network automation 'invented' now? I have been working in the system development engineering for 13+ years and we have always used automation because we wanted to save time & effort for repeatable tasks.
He’s absolutely right. We had fully-automated ISP service in early 1990’s, and numerous service providers used network automation for decades.
Worth Reading: Software Disenchantment
Found an awesome blog post describing how we’re wasting resources on incomprehensible scale. Here’s a tiny little morsel:
Only in software, it’s fine if a program runs at 1% or even 0.01% of the possible performance. Everybody just seems to be ok with it. People are often even proud about how much inefficient it is, as in “why should we worry, computers are fast enough”.
Making Sense of Software-Defined World
In mid-September I was invited to present at the vNIC 2018 event in Frankfurt, Germany. Unfortunately I wasn’t able to get there, but Zoom did a great job … and enabled me to record the talk.
Worth Watching: Machine Learning in a Nutshell
This blog post was initially sent to the subscribers of my SDN and Network Automation mailing list. Subscribe here.
What could be better than an SDN product to bring you closer to a networking nirvana? You guessed it – an SDN product using machine learning.
Want to have some fun? The next time your beloved $vendor rep drops by trying to boost his bonus by persuading you to buy the next-generation machine-learning tool his company just released, invite him to watch James Mickens’ Usenix Security Symposium keynote with you.
Leaf-and-Spine Fabric Myths (Part 2)
The next set of Leaf-and-Spine Fabric Myths listed by Evil CCIE focused on BGP:
BGP is the best choice for leaf-and-spine fabrics.
I wrote about this particular one here. If you’re not a BGP guru don’t overcomplicate your network. OSPF, IS-IS, and EIGRP are good enough for most environments. Also, don’t ever turn BGP into RIP with AS-path length serving as hop count.
Feedback: Ansible for Networking Engineers
One of my subscribers sent me a nice email describing his struggles to master Ansible:
Some time ago I started to hear about Ansible as the new power tool for network engineer, my first reaction was “What the hell is this?” I searched the web and found many blah blahs about it… until I landed on your pages.
He found Ansible for Networking Engineers material sufficient to start an automation project:
VXLAN and EVPN on Hypervisor Hosts
One of my readers sent me a series of questions regarding a new cloud deployment where the cloud implementers want to run VXLAN and EVPN on the hypervisor hosts:
I am currently working on a leaf-and-spine VXLAN+ EVPN PoC. At the same time, the systems team in my company is working on building a Cloudstack platform and are insisting on using VXLAN on the compute node even to the point of using BGP for inter-VXLAN traffic on the nodes.
Using VXLAN (or GRE) encap/decap on the hypervisor hosts is nothing new. That’s how NSX and many OpenStack implementations work.
Worth Reading: The Fragile Engineers
Ethan Banks wrote an awesome blog post on the characteristics of fragile engineers (most of them probably being expert beginners). I can’t help but ponder how often I behave like one…
Leaf-and-Spine Fabric Myths (Part 1)
Apart from the “they have no clue what they’re talking about” observation, Evil CCIE left a long list of leaf-and-spine fabric myths he encountered in the wild in a comment on one of my blog posts. He started with:
Clos fabric (aka Leaf And Spine fabric) is a non-blocking fabric
That was obviously true in the days when Mr. Clos designed the voice switching solution that still bears his name. In the original Clos network every voice call would get a dedicated path across the fabric, and the number of voice calls supported by the fabric equaled the number of alternate end-to-end paths.
Network Automation Development Environments
Building the network automation lab environment seems to be one of the early showstoppers on everyone’s network automation journey. These resources might help you get started:
- I wrote an installation script that installs the myriad dependencies needed by Ansible and NAPALM in just the right order on a Ubuntu VM (step-by-step instructions for ipSpace.net subscribers).
- Carl Buchmann open-sourced a full-blown infrastructure-as-code development environment he uses for his automation projects.
- Jaap de Vos described how he creates a Docker image containing Ansible, NAPALM and Nornir.
Hint: after setting up your environment, you might want to enroll into the Spring 2019 network automation course ;)
Network Troubleshooting Guidelines
It all started with an interesting weird MLAG bugs discussion during our last Building Next-Generation Data Center online course. The discussion almost devolved into “when in doubt reload” yammering when Mark Horsfield stepped in saying “while that may be true, make sure to check and collect these things before reloading”.
I loved what he wrote so much that I asked him to turn it into a blog post… and he made it even better by expanding it into generic network troubleshooting guidelines. Enjoy!
Don't Make a Total Mess When Dealing with Exceptions
A while ago I had the dubious “privilege” of observing how my “beloved” airline Adria Airways deals with exceptions. A third-party incoming flight was 2.5 hours late and in their infinite wisdom (most probably to avoid financial impact) they decided to delay a half-dozen outgoing flights for 20-30 minutes while waiting for the transfer passengers.
Not surprisingly, when that weird thingy landed and they started boarding the outgoing flights (now all at the same time), the result was a total mess with busses blocking each other (this same airline loves to avoid jet bridges).