One of my readers sent me a question along these lines:
Do I have to have an IBGP session between Customer Edge (CE) routers in a multihomed site if they run EBGP with the upstream provider(s)?
Let’s start with a simple diagram and a refactoring of the question:
- A multihomed site has two WAN edge (CE) routers
- Each CE-router runs EBGP with the adjacent PE-router.
- Do we need an IBGP session between CE-A and CE-B?
Our multihomed site is small enough to have a single L2 switch, and both CE-routers act as a default gateway for the attached hosts1. Now imagine a scenario where:
- CE-A receives a routing update for destination X from its upstream PE-router, but CE-B receives no corresponding update from its EBGP peer.
- A host sends a packet for X toward CE-B.
It’s obvious that CE-B should have the information that it can reach X via CE-A, and there are two ways to achieve that:
- Exchange the information over an IBGP session between CE-A and CE-B
- Redistribute EBGP information into an IGP (for example, OSPF)
As you might be running an IGP within the site and redistribute IGP information into EBGP anyway, you’ll quickly land in a two-way redistribution morass if you choose option#2. Running IBGP between CE-routers is a much better approach, and gives you the ability to have site-wide consistent routing policy. For example, you could use BGP local preference to indicate which paths should be preferred2, causing the other CE-router to prefer IBGP paths over EBGP ones.
Finally a word of caution: establishing an IBGP session between CE-routers that do not support RFC 8212 could turn your site into a transit site. Not fun if you happen to be a steel manufacturer attracting Cloudflare traffic. Make sure you have deployed outbound AS-path filters dropping transit paths on all EBGP sessions.
- Consistent routing within a multihomed site is one of the Expert Express Case Studies (disguised as an “MPLS/VPN + IPsec” question)
- We discussed redundant BGP-based Internet access in September 2022 session of ipSpace.net Design Clinic. In the same session we also discussed secure multi-homed customer BGP configurations.
- We’ll do a deep dive into securing BGP routing (including a brief history of well-known global routing FUBARs) in the Internet Routing Security webinar.
Obviously you could also use a route map to set tags or metrics in OSPF type-5 LSA when doing BGP-to-OSPF redistribution, and use that information to set administrative distance of OSPF routes (assuming your device can do something as abhorrent as that) if your primary design goal is to have infinite job security. ↩︎