Category: BGP

As I was doing the final integration tests for netlab release 1.9.0, I stumbled upon a fascinating BGP configuration quirk: where do you configure the allowas-in parameter and why?

A Bit of Theory

BGP runs over TCP, and all parameters related to the TCP session are configured for a BGP neighbor (IPv4 or IPv6 address). That includes the source interface, local AS number (it’s advertised in the per-session OPEN message that negotiates the address families), MD5 password (it uses MD5 checksum of TCP packets), GTSM (it uses the IP TTL field), or EBGP multihop (it increases the IP TTL field).

Daniel Dib asked an interesting question on LinkedIn when considering an RT5-only EVPN design:

I’m curious what EVPN provides if all you need is L3. For example, you could run pure L3 BGP fabric if you don’t need VRFs or a limited amount of them. If many VRFs are needed, there is MPLS/VPN, SR-MPLS, and SRv6.

I received a similar question numerous times in my previous life as a consultant. It’s usually caused by vendor marketing polluting PowerPoint slide decks with acronyms without explaining the fundamentals¹. Let’s fix that.

The first BGP load balancing lab exercise described the basics of EBGP equal-cost load balancing. Now for the fun part: what if you want to spread traffic across multiple links in an unequal ratio? There’s a nerd knob for that: the BGP Link Bandwidth extended community that you can test-drive in this lab exercise.

Last summer, I started a long-term project to revive the BGP labs I created in the mid-1990s. I completed the original lab exercises (BGP sessions, IBGP, local preference, MED, communities) in late 2023 but then kept going. This is how far I got in a year:

• Twenty-six deploy BGP exercises, including advanced settings like AS path manipulations, MD5 passwords and BFD, and new technologies like TCP/AO and interface EBGP sessions.
• Fifteen BGP routing policies exercises, covering the basic mechanisms as well as dirty tricks like route disaggregation
• Four load balancing exercises, from EBGP ECMP to BGP Link Bandwidth and BGP Additional Paths.
• Five challenges for everyone who got bored doing the simple stuff ;)

That completes the BGP technologies I wanted to cover. I’ll keep adding the challenge labs and advanced scenarios. Here are some ideas; if you have others, please leave a comment.

RFC 4264 defines BGP wedgies as “a class of BGP configurations for which there is more than one potential outcome, and where forwarding states other than the intended state are equally stable.” Even worse, “the stable state where BGP converges may be selected by BGP in a non-deterministic manner.”

Want to know more? You can explore a real-life BGP wedgie and fix it in the latest BGP lab exercise.

I plan to add several challenge labs using multihop EBGP sessions to the BGP labs project, including:

1. Running BGP between VMs and central BGP route servers
2. Using multihop EBGP session to send full Internet routing table to a customer without overloading the PE-router
3. Running EBGP EVPN session between loopbacks advertised with EBGP IPv4 session (🤢)

However, I would love to start with a simple use case to help engineers unfamiliar with BGP realize when they might have to use multihop EBGP sessions. Unfortunately, I can’t find one, and the scenarios where I used multihop EBGP in the past (EBGP load balancing and using a low-end router in the EBGP path, where I was effectively using the reverse application of #2 as a customer) are mostly irrelevant.

Would you have an easy-to-understand use case that is best solved with a multihop EBGP session? Please share it in the comments. Thanks a million!

I love open-source tools (and their GitHub repositories). Someone launches a cool idea, and you can dig through their source code to figure out how it works. It beats reading documentation or fixing AI hallucinations every day of the week ;)

Not too long ago, the containerlab team launched the ability to run containerlab within a free¹ container² running on GitHub, and that seemed like a perfect solution to run the BGP labs (Jeroen van Bemmel pointing me in the right direction was another significant step forward).

In late 2023, I started playing with the idea of having automated validation in netlab. The early implementation was used in BGP labs, and a user liked it so much that he opened an issue saying:

I would suggest providing netlab validate for each lab.

Numerous rounds of yak-shaving later, I merged a humongous commit that adds automated validation to these lab exercises:

Using the typical default router configurations, it can take minutes between a failure of an inter-AS link and the convergence of BGP routes. You can fine-tune that behavior with BGP timers and BFD (and still get pwned by Graceful Restart). While you can’t influence link failures, you could drain the traffic from a link before starting maintenance operations on it, and it would be a shame not to do that considering there’s a standard way to do that – the GRACEFUL_SHUTDOWN BGP community defined in RFC 8326. That’s what you’ll practice in the next BGP lab exercise.

The recent IBGP Full Mesh Between EVPN Leaf Switches blog post generated an interesting discussion on LinkedIn focused on whether we need route reflectors (in small fabrics) and whether they do more harm than good. Here are some of the highlights of that discussion, together with a running commentary.

Let’s open another juicy can of BGP worms: load balancing. In the first lab exercise, you’ll configure equal-cost load balancing across EBGP paths and tweak the “What is equal cost?” algorithm to consider just the AS path length, not the contents of the AS path.

Here’s another BGP lab challenge to start your weekend: use RIB-to-FIB filters to reduce the forwarding table size on access routers in a large Service Provider network.

If you insist on building your network with EBGP as a better IGP, make sure your implementation supports running IPv4 and IPv6 address families over EBGP sessions established between IPv6 link-local addresses (the functionality lovingly called unnumbered EBGP sessions).

Want to practice that neat trick? Check out the EBGP Sessions over IPv6 LLA Interfaces lab exercise.

Here’s another challenge for BGP aficionados: build an MPLS-based transit network without BGP running on core routers.

That should be an easy task if you configured MPLS in the past, so try to spice it up a bit:

• Use SR/MPLS instead of LDP
• Do it on a platform you’re not familiar with (hint: Arista vEOS is a bit different from Cisco IOS)
• Try to get it running on FRR containers.

Daryll Swer left a long comment describing how he designed a Service Provider network running in numerous private autonomous systems. While I might not agree with everything he wrote, it’s an interesting idea and conceptually pretty similar to what we did 25 years ago (IBGP without IGP, running across physical interfaces, with every router being a route-reflector client of every other router), or how some very large networks were using BGP confederations.

Just remember (as someone from Cisco TAC told me in those days) that “you might be the only one in the world doing it and might hit bugs no one has seen before.”