Category: IP routing

Did you ever experience an out-of-the-blue BGP session flap after you were running that peering for months? As Dmytro Shypovalov explains in his latest blog post, it’s always MTU (just kidding, of course it’s always DNS, but MTU blackholes nonetheless result in some crazy behavior).

I should have known better, but I got pulled into another stretched VLANs for disaster recovery tweetfest. Surprisingly, most of the tweets were along the lines of you really shouldn’t be doing that and that would never work well, but then I guess I was only exposed to a small curated bubble of common sense… until this gem appeared in my timeline:

Interestingly, that’s exactly how IP works:

When I still cared about CCIE certification, I was always tripped up by the weird scenario with (A) mismatched ARP and MAC timeouts and (B) default gateway outside of the forwarding path. When done just right you could get persistent unicast flooding, and I’ve met someone who reported average unicast flooding reaching ~1 Gbps in his data center fabric.

One would hope that we wouldn’t experience similar problems in modern leaf-and-spine fabrics, but one of my readers managed to reproduce the problem within a single subnet in FabricPath with anycast gateway on spine switches when someone misconfigured a subnet mask in one of the servers.

We all knew it for a long time, now it’s finally official: IP fragmentation is broken, or as the ever-so-diplomatic IETF likes to call it, IP Fragmentation is Considered Fragile.

The can we trust routing protocols series of blog posts I wrote in April 2020 (part 1, part 2, response from Jeff Tantsura) culminated in an interesting discussion with Russ White and Nick Russo now published as The Hedge Episode 43.

During the Comparing Transparent Bridging and IP Routing part of How Networks Really Work webinar I said something along the lines of:

While packets should never be reordered in transit in transparent bridging, there’s no such guarantee in IP networks, and IP applications should tolerate out-of-order packets.

One of my regular readers who designs and builds networks supporting VoIP applications disagreed with that citing numerous real-life examples.

Of course he was right, but let’s get the facts straight first:

Two weeks ago I started with a seemingly simple question:

If a BGP speaker R is advertising a prefix A with next hop N, how does the network know that N is actually alive and can be used to reach A?

… and answered it for the case of directly-connected BGP neighbors (TL&DR: Hope for the best).

Jeff Tantsura provided an EVPN perspective, starting with “the common non-arguable logic is reachability != functionality”.

Now let’s see what happens when we add route reflectors to the mix. Here’s a simple scenario:

Aldrin sent me an interesting question as a comment to one of my EVPN blog posts:

How does the network know that a VTEP is actually alive? (1) from the point of view of the control plane and (2) from the point of view of the data plane? And how do you ensure that control and data plane liveness monitoring has the same view? BFD for BGP is a possible solution for (1) but it’s not meant for 3rd party next hops, i.e. it doesn’t address (2).

Let’s stop right there (or you’ll stop reading in the next 10 milliseconds). I will also try to rephrase the question in more generic terms, hoping Aldrin won’t mind a slight detour… we’ll get back to the original question in another blog post.

After a brief overview of FRRouting suite Donald Sharp continued with a deep dive into FRR architecture, including the various routing daemons, role of Zebra and ZAPI, interface between RIB (Zebra) and FIB (Linux Kernel), sample data flow for route installation, and multi-threading in Zebra and BGP daemons.

In mid 2000s I wrote a number of articles describing various TCP/IP features. Most of them are a bit outdated, so I decided to clean up, update and repost the most interesting ones on ipSpace.net, starting with Never-Ending Story of IP Fragmentation.

The first part of that article is already online, covering MTU basics and drawbacks of IP fragmentation.

In mid-June I started another pet project - a series of webinars focused on networking fundamentals. In the first live session on June 18th we focused on identifying the challenges one has to solve when building an end-to-end networking solution, and the role of layered approach to networking.

Not surprisingly, we quickly went down the rabbit holes of computer networking history, including SCSI cables, serial connections and modems… but that’s where it all started, and some of the concepts developed at that time are still used today… oftentimes heavily morphed by recursive application of RFC 1925 Rule 11.

A few years ago we “celebrated” 512K day - the size of the full Internet routing table exceeded 512K (for whatever value of K ;) prefixes, overflowing TCAMs in some IP routers and resulting in interesting brownouts.

We’re close to exceeding 768K mark and the beware 768K day blog posts have already started appearing. While you (RFC 2119) SHOULD check the size of your forwarding table and the maximum capabilities of your hardware, the more important question should be “Why do I need 768K forwarding entries if I’m not a Tier-1 provider”

Inspired by The Zen of Python, Dinesh Dutt wrote The Zen of Routing Protocols:

Beautiful is better than ugly.

Simple is better than complex.

Complex is better than complicated.

So just because you can, don't.

Evil CCIE concluded his long list of leaf-and-spine fabric myths (more in part 1 and part 2) with a layer-2 fabric myth:

Layer 2 Fabrics can't be extended beyond 2 Spine switches. I had a long argument with a $vendor guys on this. They don't even count SPB as Layer 2 fabric and so forth.

The root cause of this myth is the lack of understanding of what layer-2, layer-3, bridging and routing means. You might want to revisit a few of my very old blog posts before moving on: part 1, part 2, what is switching, layer-3 switches and routers.

As anyone starting their journey into AWS quickly discovers, cloud is different (or as I wrote in the description of my AWS workshop you feel like Alice in Wonderland). One of the gotchas: when you link multiple routing domains (Virtual Private Clouds – the other VPC) you have to create static routing table entries on both ends. Even worse, there’s no transit VPC – you have to build a full mesh of relationships.

The correct solution to this challenge is automation: