Category: BGP

I spent a few days in a beautiful place with suboptimal Internet connectivity. The only thing I could do whenever I got bored (without waiting for the Internet gnomes to hand-carry the packets across the mountain passes) was to fix the BGP labs on a Ubuntu VM running on my MacBook Air (hint: it all works).

Big things first. I added validation to these labs:

• MD5 Passwords and GTSM
• Use BGP Timers and BFD to Speed Up BGP Convergence
• BGP Route Aggregation
• EBGP Sessions over IPv6 LLA Interfaces
• Establish an IBGP Session
• Build a Transit Network with IBGP
• Use BGP Route Reflectors
• Using No-Export Community to Filter Transit Routes

In the previous BGP load balancing lab exercise, I described the BGP Link Bandwidth attribute and how you can use it on EBGP sessions. This lab moves the unequal-cost load balancing into your network; we’ll use the BGP Link Bandwidth attribute on IBGP sessions.

Ages ago, I described how “traditional” network operating systems used the BGP Routing Information Base (BGP RIB), the system routing table (RIB), and the forwarding table (FIB). Here’s the TL&DR:

1. Routes received from BGP neighbors are stored in BGP RIB.
2. Routes redistributed into BGP from other protocols are (re)created in the BGP RIB.
3. BGP selects the best routes in BGP RIB using its convoluted set of rules.
4. Best routes from the BGP RIB are advertised to BGP neighbors
5. Best routes from the BGP RIB compete (based on their administrative distance) against routes from other routing protocols to enter the IP routing table (system RIB)
6. Routes from the system RIB are copied into FIB after their next hops are fully evaluated (a process that might involve multiple recursive lookups).

The very first BGP Communities RFC included an interesting idea: let’s tag paths we don’t want to propagate to other autonomous systems. For example, the prefixes received from one upstream ISP should not be propagated to another upstream ISP (sadly, things don’t work that way in reality).

Want to try out that concept? Start the Using No-Export Community to Filter Transit Routes lab in GitHub Codespaces.

Ever since Pawel Foremski talked about BGP Pipe @ RIPE88 meeting, I wanted to kick its tires in netlab. BGP Pipe is a Go executable that runs under Linux (but also FreeBSD or MacOS), so I could add a Linux VM (or container) to a netlab topology and install the software after the lab has been started. However, I wanted to have the BGP neighbor configured on the other side of the link (on the device talking with the BGP Pipe daemon).

I could solve the problem in a few ways:

As I was doing the final integration tests for netlab release 1.9.0, I stumbled upon a fascinating BGP configuration quirk: where do you configure the allowas-in parameter and why?

A Bit of Theory

BGP runs over TCP, and all parameters related to the TCP session are configured for a BGP neighbor (IPv4 or IPv6 address). That includes the source interface, local AS number (it’s advertised in the per-session OPEN message that negotiates the address families), MD5 password (it uses MD5 checksum of TCP packets), GTSM (it uses the IP TTL field), or EBGP multihop (it increases the IP TTL field).

Daniel Dib asked an interesting question on LinkedIn when considering an RT5-only EVPN design:

I’m curious what EVPN provides if all you need is L3. For example, you could run pure L3 BGP fabric if you don’t need VRFs or a limited amount of them. If many VRFs are needed, there is MPLS/VPN, SR-MPLS, and SRv6.

I received a similar question numerous times in my previous life as a consultant. It’s usually caused by vendor marketing polluting PowerPoint slide decks with acronyms without explaining the fundamentals¹. Let’s fix that.

The first BGP load balancing lab exercise described the basics of EBGP equal-cost load balancing. Now for the fun part: what if you want to spread traffic across multiple links in an unequal ratio? There’s a nerd knob for that: the BGP Link Bandwidth extended community that you can test-drive in this lab exercise.

Last summer, I started a long-term project to revive the BGP labs I created in the mid-1990s. I completed the original lab exercises (BGP sessions, IBGP, local preference, MED, communities) in late 2023 but then kept going. This is how far I got in a year:

• Twenty-six deploy BGP exercises, including advanced settings like AS path manipulations, MD5 passwords and BFD, and new technologies like TCP/AO and interface EBGP sessions.
• Fifteen BGP routing policies exercises, covering the basic mechanisms as well as dirty tricks like route disaggregation
• Four load balancing exercises, from EBGP ECMP to BGP Link Bandwidth and BGP Additional Paths.
• Five challenges for everyone who got bored doing the simple stuff ;)

That completes the BGP technologies I wanted to cover. I’ll keep adding the challenge labs and advanced scenarios. Here are some ideas; if you have others, please leave a comment.

RFC 4264 defines BGP wedgies as “a class of BGP configurations for which there is more than one potential outcome, and where forwarding states other than the intended state are equally stable.” Even worse, “the stable state where BGP converges may be selected by BGP in a non-deterministic manner.”

Want to know more? You can explore a real-life BGP wedgie and fix it in the latest BGP lab exercise.

I plan to add several challenge labs using multihop EBGP sessions to the BGP labs project, including:

1. Running BGP between VMs and central BGP route servers
2. Using multihop EBGP session to send full Internet routing table to a customer without overloading the PE-router
3. Running EBGP EVPN session between loopbacks advertised with EBGP IPv4 session (🤢)

However, I would love to start with a simple use case to help engineers unfamiliar with BGP realize when they might have to use multihop EBGP sessions. Unfortunately, I can’t find one, and the scenarios where I used multihop EBGP in the past (EBGP load balancing and using a low-end router in the EBGP path, where I was effectively using the reverse application of #2 as a customer) are mostly irrelevant.

Would you have an easy-to-understand use case that is best solved with a multihop EBGP session? Please share it in the comments. Thanks a million!

I love open-source tools (and their GitHub repositories). Someone launches a cool idea, and you can dig through their source code to figure out how it works. It beats reading documentation or fixing AI hallucinations every day of the week ;)

Not too long ago, the containerlab team launched the ability to run containerlab within a free¹ container² running on GitHub, and that seemed like a perfect solution to run the BGP labs (Jeroen van Bemmel pointing me in the right direction was another significant step forward).

In late 2023, I started playing with the idea of having automated validation in netlab. The early implementation was used in BGP labs, and a user liked it so much that he opened an issue saying:

I would suggest providing netlab validate for each lab.

Numerous rounds of yak-shaving later, I merged a humongous commit that adds automated validation to these lab exercises:

Using the typical default router configurations, it can take minutes between a failure of an inter-AS link and the convergence of BGP routes. You can fine-tune that behavior with BGP timers and BFD (and still get pwned by Graceful Restart). While you can’t influence link failures, you could drain the traffic from a link before starting maintenance operations on it, and it would be a shame not to do that considering there’s a standard way to do that – the GRACEFUL_SHUTDOWN BGP community defined in RFC 8326. That’s what you’ll practice in the next BGP lab exercise.

The recent IBGP Full Mesh Between EVPN Leaf Switches blog post generated an interesting discussion on LinkedIn focused on whether we need route reflectors (in small fabrics) and whether they do more harm than good. Here are some of the highlights of that discussion, together with a running commentary.