Dynamic Negotiation of BGP Capabilities
I wanted to write a blog post explaining the intricacies of Advertisement of Multiple Paths in BGP, got into a yak-shaving exercise when discussing the need to exchange BGP capabilities to enable this feature, and decided to turn it into a separate prerequisite blog post. The optimal path selection with BGP AddPath post is coming in a few days.
The Problem
Whenever you want to use BGP for something else than simple IPv4 unicast routing the BGP neighbors must agree on what they are willing to do – be it multiprotocol extensions and individual additional address families, graceful restart, route refresh… (IANA has the complete BGP Capability Codes registry).
BGP capabilities were designed to be negotiated at the start of a BGP session and advertised within the BGP OPEN message, which means that you have to tear down BGP sessions whenever a new functionality (like BGP Add Path) or address family is configured.
Some BGP implementations expect you to tear down the sessions yourself which makes for fun troubleshooting if you never heard about BGP capability exchange – nothing happens until you clear the BGP session, and then all of a sudden everything works1.
Other implementations are more aggressive2 and automatically reset BGP sessions whenever you configure something that requires a renegotiation of BGP capabilities3. Some go as far as calling that Hammer-of-Thor approach Dynamic Capability Exchange.
The Solution
Then there’s the right way of doing things: Dynamic Capability for BGP-4 draft describes a new CAPABILITY BGP message that can be used to renegotiate BGP capabilities without bringing down a BGP session. For whatever reason, that document has been sitting in draft status for almost a decade – looks like everyone loves bringing down their BGP sessions for no good reason.
Is There Anybody Out There?
I know I’m missing tons of details and I’d love to get your feedback. I hope to be able to end with a table describing how various BGP implementations work, so please leave a comment:
- How does the platform you use handle changes in BGP address families or other BGP features like additional paths?
- Is the tear down of BGP session on a configuration change that requires capability renegotiation: nonexistent, automatic or configurable?
- Did anyone implemented Dynamic Capability for BGP-4 draft? A quick search indicates it might be implemented by NX-OS, but the documentation claims something completely different (automatic reset of BGP sessions).
-
Makes you wonder whether there’s some truth behind the did you reload the box question you get from level-1 support. ↩︎
-
I could get snarky and claim they are more aggressive to lower the vendor support costs, but vendors would never risk the stability of your network to reduce their costs, would they? ↩︎
-
At least some implementations of dynamic capabilities are configurable. ↩︎
The draft about BGP dynamic capability has been updated last month after being dead. Is that something you triggered? The changes include something about add-path.
I think that has more to do with there being an IETF meeting. I found that draft in an overview article by Geoff Huston.
https: //www.juniper.net/documentation/en_US/junos/topics/topic-map/bgp-sessions.html
What a juicy can of worms. Thanks for sharing!
Reminds me of BGP mutisession
https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-multisession
Yeah, I found that one once I started digging into BGP-over-QUIC -- it's so much fun to solve a problem (real or perceived) with shiny new toys instead of doing it the right way a decade ago ;)
All of a sudden, multi-session makes perfect sense, while that draft was languishing in the IDR WG for over 17 years.
We were actually adding a new Afi/safi to an existing peering between 7200 and 7600, once added there was no BGP flap (surprisingly) thanks to this multisession being enabled by default on those : back in the 2008 it was like wow,this is so cool for the future!
Ah, so it's been implemented in Cisco IOS (and probably others as well). Thanks a million for that tidbit, time to do some testing...
At that time the IOS versions had that enabled by default (not sure if it was not a bug), later on it moved under
"neighbor $X transport multi-session"
so needed to be enabled explicitly.
Thank you Ivan for the blog and all the info you share. Regards, Alex
Every version of IOS, IOS XE, and ASA OS I've used requires manual clearing of BGP neighbors to renegotiate capabilities, so imagine my surprise when I gave our firewall team the green light to turn on graceful restart on their FPR HA pair via FMC which would be followed by bouncing BGP in a change window, only to see the BGP neighbor relationships all reset as soon as the config change was pushed.
I really appreciate the consistency, Cisco.