CE-to-CE IBGP Session in a Multihomed Site

One of my readers sent me a question along these lines:

Do I have to have an IBGP session between Customer Edge (CE) routers in a multihomed site if they run EBGP with the upstream provider(s)?

Let’s start with a simple diagram and a refactoring of the question:

  • A multihomed site has two WAN edge (CE) routers
  • Each CE-router runs EBGP with the adjacent PE-router.
  • Do we need an IBGP session between CE-A and CE-B?
Please note that it doesn’t matter if we’re talking about an MPLS/VPN- or a redundant Internet access deployment. There’s no difference between the two scenarios from the CE-router perspective.

Our multihomed site is small enough to have a single L2 switch, and both CE-routers act as a default gateway for the attached hosts1. Now imagine a scenario where:

  • CE-A receives a routing update for destination X from its upstream PE-router, but CE-B receives no corresponding update from its EBGP peer.
  • A host sends a packet for X toward CE-B.

It’s obvious that CE-B should have the information that it can reach X via CE-A, and there are two ways to achieve that:

  1. Exchange the information over an IBGP session between CE-A and CE-B
  2. Redistribute EBGP information into an IGP (for example, OSPF)

As you might be running an IGP within the site and redistribute IGP information into EBGP anyway, you’ll quickly land in a two-way redistribution morass if you choose option#2. Running IBGP between CE-routers is a much better approach, and gives you the ability to have site-wide consistent routing policy. For example, you could use BGP local preference to indicate which paths should be preferred2, causing the other CE-router to prefer IBGP paths over EBGP ones.

Finally a word of caution: establishing an IBGP session between CE-routers that do not support RFC 8212 could turn your site into a transit site. Not fun if you happen to be a steel manufacturer attracting Cloudflare traffic. Make sure you have deployed outbound AS-path filters dropping transit paths on all EBGP sessions.

More Details

  1. Ignoring for the moment the intricacies of first-hop redundancy protocols and ICMP redirects↩︎

  2. Obviously you could also use a route map to set tags or metrics in OSPF type-5 LSA when doing BGP-to-OSPF redistribution, and use that information to set administrative distance of OSPF routes (assuming your device can do something as abhorrent as that) if your primary design goal is to have infinite job security. ↩︎

Latest blog posts in Site and Host Multihoming series


  1. "causing the other CE-router to prefer IBGP paths over IBGP ones" Must be a typo

  2. Ivan,

    I hope this IBGP neighborship will also help in forming an SDWAN Tunnel from CE-B to remote sites using ISP-A and from CE-A to remote sites using ISP-B?

    Thanks, Viki

  3. In such a small deployment I assume that both ISPs offer transit, so that both CEs would get a default route from their upstream.

    In this case I would not iBGP the CEs together but have HSRP running on the two CEs and track the uplink (interface and/of BGP session) to determine the active gateway.

    It's simpler, less memory intensive the convergence will be faster and you don't have to care about the GRT.

    And if you wanted to IBGP them anyway, I would put a route-map on it to only exchange the default route from the upstreams, so that both CEs have a 0/0 route with different distance. The only thing I don't understand is in which failure scenario traffic would end up on a CE without an active BGP uplink.

    1. pvh, not a good idea. Sometimes your ISP has routing failures during which all physical links and BGP sessions remain up, but no routes get advertised / exchanged. The only correct option is to establish an IBGP overlay, even if using FHRP. As the "site" grows it may require route reflection.

Add comment