Building network automation solutions

9 module online course

Start now!

BGP Labeled Unicast on Cisco IOS

While researching the BGP RFCs for the Three Dimensions of BGP Address Family Nerd Knobs, I figured out that the BGP Labeled Unicast (BGP-LU, advertising MPLS labels together with BGP prefixes) uses a different address family. So far so good.

Now for the intricate bit: a BGP router might negotiate IPv4 and IPv4-LU address families with a neighbor. Does that mean that it’s advertising every IPv4 prefix twice, once without a label, and once with a label? Should that be the case, how are those prefixes originated and how are they stored in the BGP table?

As always, the correct answer is “it depends”, this time on the network operating system implementation. This blog post describes Cisco IOS behavior, a follow-up one will focus on Arista EOS.

The Lab

Whenever I get a question along the lines “what would happen if…” I always ask the sender “… and did you consider testing that in a lab?"1. We’ll test BGP-LU behavior in a simple lab with three autonomous systems and a route reflector in the central autonomous system.

BGP sessions in the BGP-LU lab

BGP sessions in the BGP-LU lab

Labeled Unicast IPv4 address family (IPv4-LU) will be enabled on all BGP sessions apart from the PE1-CE1 session, so we’ll be able to observe BGP-LU behavior on IBGP and EBGP sessions, as well as propagation of information (or lack thereof) between unlabeled and labeled address families.

The lab topology file is available on GitHub; you’ll need netsim-tools release 1.2 to deploy the lab2.

Cisco IOS Behavior

Cisco IOS configuration treats labeled unicast as an add-on to IPv4 or IPv6 address family – instead of activating a neighbor in a new address family, you add send-label parameter to a neighbor within IPv4 or IPv6 address family.

BGP-LU configuration on Cisco IOS
router bgp 65000
 bgp router-id
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor remote-as 65000
 neighbor description rr
 neighbor update-source Loopback0
 neighbor remote-as 65101
 neighbor description ce1
 address-family ipv4
  network mask
  neighbor activate
  neighbor send-community both
  neighbor next-hop-self
  neighbor send-label explicit-null
  neighbor activate
  neighbor send-community

When you configure neighbor send-label, Cisco IOS tries to negotiate two address families (IP and IP-LU) on the BGP session:

BGP address families negotiated between PE1 and RR
pe1#show ip bgp nei | section capabilities
  Neighbor capabilities:
    Route refresh: advertised and received(new)
    Four-octets ASN Capability: advertised and received
    Address family IPv4 Unicast: advertised and received
    ipv4 MPLS Label capability: advertised and received
    Enhanced Refresh Capability: advertised and received
    Multisession Capability:
    Stateful switchover support enabled: NO for session 1

Even though the routers negotiated IPv4 and IPv4-LU address families on the BGP session, the IPv4 BGP updates are sent only within the IPv4-LU address family. IPv4 address family is not used at all.

BGP updates between PE1 and CE1 (both running Cisco IOS)
pe1#debug ip bgp all updates in
BGP updates debugging is on for neighbor (inbound) for all address families
pe1#clear ip bgp
%BGP-5-ADJCHANGE: neighbor Up
BGP(0): rcvd UPDATE w/ attr: nexthop, origin i, localpref 100, metric 0, originator, clusterlist, merged path 65102, AS_PATH
BGP(0): rcvd, label 22
BGP(0): rcvd UPDATE w/ attr: nexthop, origin i, localpref 100, metric 0, originator, clusterlist
BGP(0): rcvd, label 0
BGP(0): rcvd UPDATE w/ attr: nexthop, origin i, localpref 100, metric 0
BGP(0): rcvd, label 0

The unlabeled and labeled BGP prefixes are stored in the same BGP RIB. Labels are automatically assigned to all prefixes that are advertised over Labeled Unicast address families.

Consider the labels assigned to BGP prefixes on PE1 (the In Label column contains labels assigned by the local router):

BGP labels on PE1
pe1#show ip bgp labels
   Network          Next Hop      In label/Out label         imp-null/nolabel        nolabel/exp-null        nolabel/exp-null        21/nolabel        nolabel/22
  • No label is assigned to, and because these prefixes aren’t advertised over any BGP-LU sessions – they were received over IBGP session from the route reflector and are advertised over EBGP session to CE1 (which has not activated the IPv4-LU AF).
  • A label is assigned to even though that prefix was not received over IPv4-LU address family.

The situation is slightly different on PE2 that has BGP-LU sessions with RR and CE2:

BGP labels on PE2
pe2#show ip bgp labels
   Network          Next Hop      In label/Out label        18/exp-null         imp-null/nolabel        16/exp-null        23/21        22/exp-null

Going from BGP RIB to IP routing and forwarding tables, there’s no difference between labeled and unlabeled prefixes apart from the label stack associated with the prefix advertised over IPv4-LU address family.

For example, the BGP-LU label for (22) is combined with an LDP label for PE2 (16) on PE1 to get a label stack in the forwarding table:

MPLS label stack for CE2 loopback on PE1
pe1#show mpls forwarding-table detail
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
Label      Label      or Tunnel Id     Switched      interface
None       22      0             Gi0/2
	MAC/Encaps=14/22, MRU=1496, Label Stack{16 22}
	5254004057F9525400D3B3948847 0001000000016000
	No output feature configured

Summary: Cisco IOS

  • Labeled unicast is configured as yet another parameter on a BGP neighbor.
  • The routers negotiate two address families, but use only the labeled address family to send the updates – a nice optimization and a fallback mechanism in case the remote router does not support labeled unicast.
  • There’s a single BGP RIB for labeled and unlabeled prefixes.
  • Labeled and unlabeled prefixes are inserted into IP routing- and forwarding tables.

  1. … often followed by no reply whatsoever. Some people think they can use bloggers as Free Encyclopedia of Useless Trivia and walk away when asked to do a bit of homework first. ↩︎

  2. netsim-tools release 1.2 is available as a beta release. Follow the installation instructions, and upgrade the netsim-tools package with pip3 install --upgrade --pre netsim-tools↩︎


  1. Having one RIB for labeled and unlabeled unicast on IOS has caused weird interop problem in the field. Especially since most other implementations (even Cisco IOS-XR) have different RIB and negotiate labeled and unlabeled SAFI separately.

    Take for example 6PE - remote PE expects an update with :FFFF: nexthop + MPLS label. Unlabeled IPv6 SAFI is not used between PE so it will probably not be negotiated. IOS code has some checks which tell whether to attach label or not, depending on nexthop format, whether nexthop was changed etc. Sometimes it might decide to send an unlabeled update over a labeled unicast session.

    It can get even worse if the peer is pre-RFC7606, so it might just drop the BGP session with all AF because of this.

    1. Been there, done that. Indeed interop can be tricky: IOS, IOS-XE, IOS-XR, add other vendors in the mix. It can also change between software versions! Thus, lab validation required. I remember outages caused by prefixes stuck on PE, which were not withdrawn correctly.

      In the past, XR did not allow having both AF configured on the same neighbor. While it worked, you had a syslog telling you this was not supported (and this could be fine if you 100% knew what you were doing, i.e controlled prefix filtering and label allocation). The best practice was to have 2 BGP sessions: 1 for U, 1 for LU. But for major SP with thousands of PEs, it means new loopbacks to provision (and advertise).

      This behavior was changed in XR 6.2.3 I believe:

      "For default VRF, starting from Cisco IOS XR Software Release 6.2.x, both IPv4 Unicast and IPv4 Labeled-unicast address families are supported under the same neighbor"

      And this triggered additional problems which were mitigated with... BGP knobs! I know Ivan is a fan of weird BGP knobs, so here you are:

      update in merge safi unicast labeled-unicast: I think this one was used to withdraw both entries, independently which SAFI we received the BGP withdraw. Introduced in 6.6.x I believe. It was hidden. It was later renamed to "update in labeled-unicast equivalent" in 7.1.1

      advertise local-labeled-route safi-unicast disable: disable the advertisement of a route with a local label to peers on the unicast SAFI

      Both of them are now documented:

      Voila. This is what I remember from a 2018 escalation. And you now have material to write another post ;-) Fred

  2. FRR uses the same approach. BGP-LU routes live in BGP rib. No duplication.

Add comment