Category: BGP

In the previous blog post in this series, I described why it’s (almost) impossible to implement unequal-cost multipathing for anycast services (multiple servers advertising the same IP address or range) with OSPF. Now let’s see how easy it is to solve the same challenge with BGP DMZ Link Bandwidth attribute.

I didn’t want to listen to the fan noise generated by my measly Intel NUC when simulating a full leaf-and-spine fabric, so I decided to implement a slightly smaller network:

Here’s one of the major differences between Facebook and Google: one of them publishes research papers with helpful and actionable information, the other uses publications as recruitment drive full of we’re so awesome but you have to trust us – we’re not sharing the crucial details.

Recent data point: Facebook published an interesting paper describing their data center BGP design. Absolutely worth reading.

Just in case you haven’t realized: Petr Lapukhov of the RFC 7938 fame moved from Microsoft to Facebook a few years ago. Coincidence? I think not.

In the previous blog posts in this series, we explored whether we need addresses on point-to-point links (TL&DR: no), whether it’s better to have interface or node addresses (TL&DR: it depends), and why we got unnumbered IPv4 interfaces. Now let’s see how IP routing works over unnumbered interfaces.

The Challenge

A cursory look at an IP routing table (or at CCNA-level materials) tells you that the IP routing table contains prefixes and next hops, and that the next hops are IP addresses. How should that work over unnumbered interfaces, and what should we use for the next-hop IP address in that case?

Ever since draft-lapukhov was first published almost a decade ago, we all knew BGP was the only routing protocol suitable for data center networking… or at least Thought Leaders and vendor marketers seem to be of that persuasion.

After I created the Segment Routing lab to test the relationship between Node Segment ID (SID) and MPLS labels, I was just a minor step away from testing BGP-free core with SR-MPLS.

I added two nodes to my lab setup, this time using IOSv as those nodes need nothing more than EBGP support (and IOSv is tiny compared to IOS XE on CSR):

When I’d first seen BGP-LS I immediately thought: “it would be cool to use this to fetch link state topology data from the network and build a graph out of it”. In those days the only open-source way I could find to do it involved Open DayLight controller’s BGP-LS-to-REST-API converter, and that felt like deploying an aircraft carrier to fly a kite.

Things have improved dramatically since then. In Visualizing BGP-LS Tables, HB described how he solved the challenge with GoBGP, gRPC interface to GoBGP, and some Python code to parse the data and draw the topology graph with NetworkX. Enjoy!

This podcast introduction was written by Nick Buraglio, the host of today’s podcast.

In today’s evolving landscape of whitebox, brightbox, and software routing, a small but incredibly comprehensive routing platform called FreeRTR has quietly been evolving out of a research and education service provider network in Hungary. 

Kevin Myers of IPArchitechs brought this to my attention around March of 2019, at which point I went straight to work with it to see how far it could be pushed.

Corey Quinn mentioned me in a tweet linking to AWS announcement that they are the biggest user of BGP RPKI (by the size of signed address space) worldwide. Good for them – I’m sure it got their marketing excited. It’s also trivial to do once you have the infrastructure in place. Just saying…

On a more serious front: how important is RPKI and what misuses can it stop?

If you’ve never heard of RPKI, the AWS blog post is not too bad, Nick Matthews wrote a “look grandma, this is how it works” version in 280-character installments, and you should definitely spend some time exploring MANRS resources. Here’s a short version for differently-attentive ;))

TL&DR: If you run multiple IGP protocols in your network, and add BGP on top of that, you might get the results you deserve. Even better, the results are platform-dependent.

One of my readers sent me a link to an interesting scenario described by Jeremy Filliben that results in totally unexpected behavior when using too many routing protocols in your network (no surprise there).

Imagine a network in which two edge routers advertise the same (external) BGP prefix. All other things being equal, it would make sense that other routers in the same autonomous system should use the better path out of the autonomous system. Welcome to the final tie-breaker in BGP route selection process: IGP metric.

We had the usual gloomy December weather during the end-of-year holidays, and together with the partial lockdown (with confusing ever-changing rules only someone in Balkans could dream up) it managed to put me in OCD mood… and so I decided to remove broken links from the old blog posts.

While doing that I figured out how fragile our industry is – I encountered a graveyard of ideas and products that would make Google proud. Some of those blog posts were removed, I left others intact because they still have some technical merits, and I made sure to write sarcastic update notices on product-focused ones. Consider those comments Easter eggs… now go and find them ;))

Continuing my archeological explorations, I found a dusty bag of old QoS content:

• Queuing Principles
• QoS Policing
• Traffic Shaping
• Impact of Transmit Ring Size (tx-ring-limit)
• FIFO Queuing
• Fair Queuing in Cisco IOS

I kept digging and turned out a few MPLS, BGP, and ADSL nuggets worth saving:

Remember my BGP route selection rules are a clear failure of intent-based networking paradigm blog post? I wrote it almost three years ago, so maybe you want to start by rereading it…

Making long story short: every large network is a unique snowflake, and every sufficiently convoluted network architect has unique ideas of how BGP route selection should work, resulting in all sorts of crazy extended BGP communities, dozens if not hundreds of nerd knobs, and 2000+ pages of BGP documentation for a recent network operating system (no, unfortunately I’m not joking).

More than a decade ago I published tons of materials on a web site that eventually disappeared into digital nirvana, leaving heaps of broken links on my blog. I decided to clean up those links, and managed to save some of the vanished content from the Internet Archive:

• OSPF Flooding Filters in Hub-and-Spoke Environments
• Implicit and Explicit Null Label in MPLS networks
• Default Routes in BGP
• Filter Excessively Prepended BGP Paths

I also updated dozens of blog posts while pretending to be Indiana Jones, including:

Did you ever experience an out-of-the-blue BGP session flap after you were running that peering for months? As Dmytro Shypovalov explains in his latest blog post, it’s always MTU (just kidding, of course it’s always DNS, but MTU blackholes nonetheless result in some crazy behavior).

Darren O’Connor put together a BGP looking glass with web GUI. Nothing fancy so far… but he also offers REST API interface (because REST API sounds so much better than HTTP).

The REST API calls return text results, so you can use them straight in a Bash script. For example, here’s a simple script to print a bunch of details about your current IP address: