Leaf-and-Spine Fabrics Between Theory and Reality
I’m always envious of how easy networking challenges seem when you’re solving them in PowerPoint, for example, when an innovation specialist explains how scalability works in leaf-and-spine fabrics in a LinkedIn comment:
One of the main benefits of a CLOS folded spine topology is the scale out spine where you can scale out the number of spine nodes increasing your leaf-spine n-way ECMP as well as minimizing the blast radius with the more spine nodes the more redundancy and resiliency.
Isn’t that wonderful? If you need more bandwidth, sprinkle the magic spine powder on your fabric, add water, and voila! Problem solved. Also, it looks like adding spine switches reduces the blast radius. Who would have known?
Test DHCP Relaying with netlab
After figuring out how DHCP relaying works, I decided to test it out in a lab. netlab has no DHCP configuration module (at the moment); the easiest way forward seemed to be custom configuration templates combined with a few extra attributes.
Lab Topology
This is how I set up the lab:
Worth Reading: Putting Large Language Models in Context
Another take on “what are large language models and what can we expect from them,” this time by Bruce Davie: Putting Large Language Models in Context:
My approach, at least for now, is to treat these LLM-based systems as very large, efficient collections of matchboxes–and keep working in my chosen field of networking.
Worth Reading: The War on Expertise
Jeff McLaughlin published an excellent blog post perfectly describing what we’ve been experiencing for decades: the war on expertise.
On one hand, the “business owners” force us to build complex stuff because they think they know better, on the other they blame people who know how to do it for the complex stuff that happens as the result of their requirements:
I am saying that we need to stop blaming complexity on those who manage to understand it.
Enjoy!
Video: SD-WAN Backend Architecture
After describing the SD-WAN reference design, Pradosh Mohapatra focused on individual components of an SD-WAN solution, starting with the backend architecture.
DHCP Relaying Details
Chinar Trivedi asked an interesting question about DHCP relaying in VXLAN/EVPN world on Twitter and my first thought was “that shouldn’t be hard” but when I read the first answer that turned into “wait a minute, how exactly does DHCP relaying works?”
I’m positive there’s a tutorial out there somewhere, but I decided to go back to the sources of wisdom: the RFCs. It turned out to be a long walk down the IETF history lane.
New: Anycast Resource Page
I wrote two dozen blog posts describing IP anycast concepts, from first-hop anycast gateways to anycast between DNS servers and global anycast (as used by large web properties), but never organized them in any usable form.
That’s fixed: everything I ever wrote about anycast is nicely structured on the new Anycast Resources page.
Dynamic MAC Learning: Hardware or CPU Activity?
An ipSpace.net subscriber sent me a question along the lines of “does it matter that EVPN uses BGP to implement dynamic MAC learning whereas in traditional switching that’s done in hardware?” Before going into those details, I wanted to establish the baseline: is dynamic MAC learning really implemented in hardware?
Hardware-based switching solutions usually use a hash table to implement MAC address lookups. The above question should thus be rephrased as is it possible to update the MAC hash table in hardware without punting the packet to the CPU? One would expect high-end (expensive) hardware to be able do it, while low-cost hardware would depend on the CPU. It turns out the reality is way more complex than that.
netlab: Change Stub Networks into Loopbacks
One of the least-documented limitations of virtual networking labs is the number of network interfaces a virtual machine could have. vSphere supports up to 10 interfaces per VM, the default setting for vagrant-libvirt is eight, and I couldn’t find the exact numbers for KVM. Many vendors claim their KVM limit is around 25; I was able to bring up a Nexus 9300v device with 40 adapters.
Anyway, a dozen interfaces should be good enough if you’re building a proof-of-concept fabric, but it might get a bit tight if you want to emulate plenty of edge subnets.
Video: Getting Started with netlab
After explaining how netlab fits into the virtual lab orchestration picture and what exactly it can do, let’s focus on what’s the easiest way to get started.
The next video in the Using netlab to Build Networking Labs series describes:
- Typical deployment scenarios: VirtualBox on Windows or MacOS, or libvirt/KVM on a Linux server or a virtual machine (running on Windows or MacOS).
- Hardware and software requirements
- Behind-the-scene operations performed by netlab create, netlab initial and netlab up commands.
History of IP TTL in EBGP Sessions
Chris Parker wrote a wonderful blog post going deep into the weeds on how EBGP sessions use IP TTL and why we need multihop EBGP sessions between adjacent devices. However, he couldn’t find a source explaining why early BGP implementations decided to use IP TTL set to one on EBGP sessions:
If there’s a source on the internet that explains when it was decided that EBGP should use a TTL of 1, I can’t find it. I can’t even find it in any RFC. I looked in the RFC for BGP v4, and went all the way back to BGP v1. None of these documents contain the text “TTL or “time to live” or “time-to-live.” It’s not even in the RFC for EGP, back in 1984.
Feedback: Microsoft Azure Networking
Numerous networking engineers found my cloud webinars (AWS, Azure) useful when preparing for a cloud migration project. Here’s what one of them wrote:
We are beginning to migrate some of our offerings to Microsoft Azure and I need to get up to speed with Azure products. I found this webinar very informative, and Ivan explained the concepts in a clear manner and easy to follow along. I would recommend watching these webinars and then read Microsoft documentation to get a thorough understanding.
Want to have some hands-on work sprinkled on top of that? You’ll find deployment examples in the Networking in Public Clouds GitHub repository.
Alternatives to IBGP within Multihomed Sites
Two weeks ago I explained why you might want to run IBGP between CE-routers on a multihomed site. One of the blog readers didn’t like my ideas:
In such a small deployment I assume that both ISPs offer transit, so that both CEs would get a default route from their upstream.
In this case I would not iBGP the CEs together but have HSRP running on the two CEs and track the uplink (interface and/of BGP session) to determine the active gateway.
Let’s see what could possibly go wrong with that design.
Suspending Devices in netlab Labs
A networking engineer tired of waiting for network devices to start sent me this question:
Can you suspend VMs in netlab? I use this trick in vSphere with CSR1Kv.
TL&DR: Maybe. Probably not.
Video: Packet Buffers in Data Center ASICs
A few years ago, we were fortunate enough to have Pete Lumbis talking about ASICs for Networking Engineers as part of the Data Center Fabric Architectures webinar.
One of the topics he couldn’t possibly skip was the question of how many packet buffers one needs in a data center switch.