Category: BGP

One of my subscribers found an unusual BGP specimen in the wild:

• It was a small site with two core switches and a WAN edge router
• The site had VPN concentrators running in virtual machines
• The WAN edge router was running BGP across WAN IPsec tunnels
• The VPN concentrators were running BGP with core switches.

So far so good, and kudos to whoever realized BGP is the only sane protocol to run between virtual machines and network core. However, the routing in the network core was implemented with EBGP sessions between the three core devices, and my subscriber thought the correct way to do it would be to use IBGP and OSPF.

BGP zombies are routes in the BGP table that refuse to disappear even though they should have been long gone. Recent measurements estimate between 0.5% and 1.5% of all routes in the global BGP table are zombies, which sounds crazy – after all, BGP is supposed to be pretty reliable.

Daryll Swer identified one potential source – Juniper routers do not revoke suppressed aggregated prefixes – and documented it in Navigating a BGP zombie outbreak on Juniper routers.

I got this question from one of my readers:

Why are OSPF and BGP are more complex than STP from a designer or administrator point of view? I tried everything to come to a conclusion but I couldn’t find a concluded answer, ChatGPT gave a circular loop answer.

There are numerous reasons why a protocol, a technology or a solution might be more complex than another seemingly similar one (or as Russ White would have said, “if you haven’t found the tradeoffs, you haven’t looked hard enough”):

One of my subscribers sent me this question:

I’m being asked to enter a working group on RPKI and route origination. I’m doing research, listening to Jeff Tantsura, who seems optimistic about taking steps to improve BGP security vs Geoff Huston, who isn’t as optimistic. Should I recommend to the group that the application security is the better investment?

You need both. RPKI is slowly becoming the baseline of global routing hygiene (like washing hands, only virtual, and done once every blue moon when you get new IP address space or when the certificates expire). More and more Internet Service Providers (including many tier-1 providers) filter RPKI invalids thus preventing the worst cases of unintentional route leaks.

I wanted to include a few examples of BGP bugs causing widespread disruption in the Network Security Fallacies presentation. I tried to find what happened when someone announced beacon prefixes with unknown optional transitive attributes (which should have been passed without complaints but weren’t) without knowing when it happened or who did it.

Trying to find the answer on Google proved to be a Mission Impossible – regardless of how I structured my query, I got tons of results that seemed relevant to a subset of the search words but nowhere near what I was looking for. Maybe I would get luckier with a tool that’s supposed to have ingested all the world’s knowledge and seems to (according to overexcited claims) understand what it’s talking about.

A few weeks ago I described why EBGP TCP packets have TTL set to one (unless you configured EBGP multihop). Although some people claim that (like NAT) it could be a security feature, it’s not a good one. Generalized TTL Security Mechanism (GTSM, described in RFC 5082) is much better.

Most BGP implementations set TTL field in outgoing EBGP packets to one. That prevents a remote intruder that manages to hijack a host route to an adjacent EBGP peer from forming a BGP session as the TCP replies get lost the moment they hit the first router in the path.

Chris Parker wrote a wonderful blog post going deep into the weeds on how EBGP sessions use IP TTL and why we need multihop EBGP sessions between adjacent devices. However, he couldn’t find a source explaining why early BGP implementations decided to use IP TTL set to one on EBGP sessions:

If there’s a source on the internet that explains when it was decided that EBGP should use a TTL of 1, I can’t find it. I can’t even find it in any RFC. I looked in the RFC for BGP v4, and went all the way back to BGP v1. None of these documents contain the text “TTL or “time to live” or “time-to-live.” It’s not even in the RFC for EGP, back in 1984.

Two weeks ago I explained why you might want to run IBGP between CE-routers on a multihomed site. One of the blog readers didn’t like my ideas:

In such a small deployment I assume that both ISPs offer transit, so that both CEs would get a default route from their upstream.

In this case I would not iBGP the CEs together but have HSRP running on the two CEs and track the uplink (interface and/of BGP session) to determine the active gateway.

Let’s see what could possibly go wrong with that design.

One of my readers sent me a question along these lines:

Do I have to have an IBGP session between Customer Edge (CE) routers in a multihomed site if they run EBGP with the upstream provider(s)?

Let’s start with a simple diagram and a refactoring of the question:

I like using Cisco IOS for my routing protocol virtual labs¹. It uses a trivial amount of memory² and boots relatively fast. There was just one thing that kept annoying me: Cisco IOS release 15.x takes forever to install local routes in the BGP table and even longer to select the best routes and propagate them³.

I finally found the culprit: bgp update-delay nerd knob. Here’s what the documentation has to say about it:

The ipSpace.net Design Clinic has been running for a bit over than a year. We covered tons of interesting technologies and design challenges, resulting in over 13 hours of content (so far), including several BGP-related discussions:

• BGP route servers
• Redundant BGP-Based Internet Access
• Secure BGP Configuration on Customer Routers
• Enterprise WAN Routing Design

All the Design Clinic discussions are available with Standard or Expert ipSpace.net Subscription, and anyone can submit new design/discussion challenges.

Every time I mention unnumbered BGP sessions in a webinar, someone inevitably asks “and how exactly does that work?” I always replied “gee, that’s a blog post I should write one of these days,” and although some readers might find it long overdue, here it is ;)

We’ll work with a simple two-router lab with two parallel unnumbered links between them. Both devices will be running Cumulus VX 4.4.0 (FRR 8.4.0 container generates almost identical printouts).

Bela Varkonyi left two intriguing comments on my Leave BGP Next Hops Unchanged on Reflected Routes blog post. Let’s start with:

The original RR design has a lot of limitations. For usual enterprise networks I always suggested to follow the topology with RRs (every interim node is an RR), since this would become the most robust configuration where a link failure would have the less impact.

He’s talking about the extreme case of hierarchical route reflectors, a concept I first encountered when designing a large service provider network. Here’s a simplified conceptual diagram (lines between boxes are physical links as well as IBGP sessions between loopback interfaces):

Here’s the last question I’ll answer from that long list Daniel Dib posted weeks ago (answer to Q1, answer to Q2).

I am trying to understand what made the BGP designers decide that RR should not change the BGP Next Hop for IBGP-learned routes.

I’m always in a bit of a bind when I get an invitation to speak at a security conference (after all, I know just enough about security to make a fool of myself), but when the organizers of the DEEP Conference invited me to talk about Internet routing security I simply couldn’t resist – the topic is dear and near to my heart, and I planned to do a related webinar for a very long time.

Even better, that conference would have been my first on-site presentation since the COVID-19 craze started, and I love going to Dalmatia (where the conference is taking place). Alas, it was not meant to be – I came down with high fever just days before the conference and had to cancel the talk.