Category: MPLS VPN

After discussing the basics of MPLS and LDP in our chat, Seamus Gilchrist and myself focused on a concept that perplexes many networking engineers entering the MPLS world: the relationship between Forward Equivalence Classes (FEC), LDP and BGP.

One of my readers sent me an interesting challenge:

We have two MPLS providers sending us default routes and it seems like whenever we have problem with SP1 our failover is not happening properly and actually we have to go in manually and influence our traffic to forward via another path.

Welcome to the wondrous world of byzantine routing failures ;)

MPLS bottom-of-stack bit confused one of my readers. In particular, he had a problem with the part where the egress MPLS Label Switch Router (LSR) should go from labeled (MPLS) to unlabeled (IPv4, IPv6) packets and had to figure out what was in the packet.

Someone recently sent me this scenario:

Our CIO has recently told us that he wants to get rid of MPLS because it is too costly and is leaning towards big Internet lines running IPSEC VPNs to connect the whole of Africa.

He was obviously shopping around for free advice (my friend Jeremy Stretch posted his answers to exactly the same set of questions not so long ago); here are the responses I wrote to his questions:

Paul Unbehagen made an interesting claim when presenting Avaya network built for Sochi Olympics during a recent Tech Field Day event: “we didn’t need MPLS or BGP to implement L2- and L3VPN. It was all done with SPB and IS-IS.”

During my ExpertExpress engagements with engineers building multi-tenant cloud infrastructure I often get questions along the lines of “How do I integrate my public IaaS cloud with my MPLS/VPN WAN?” Here are a few ideas.

A good friend of mine sent me an interesting question:

When I configure mpls ip on an interface, will all packets on that interface be labeled, or just the MPLS/VPN packets received through VRFs? I always assumed that stuff in the global routing table just got forwarded as IP packets without any labels.

Well, that’s not how MPLS works (at least not in its default incarnation on Cisco IOS).

One of the Expert Express sessions focused on an MPLS/VPN-based WAN network using OSPF as the routing protocol. The customer wanted to add DMVPN-based backup links and planned to retain OSPF as the routing protocol. Not surprisingly, the initial design had all sorts of unexpectedly complex kludges (see the case study for more details).

Having a really smart engineer on the other end of the WebEx call, I had to ask a single question: “Why don’t you use BGP everywhere” and after a short pause got back the expected reply “wow ... now it all makes sense.”

Have you ever tried to implement a large-scale DMVPN or MPLS/VPN network using BGP as the routing protocol? If you tried to stitch more than ~1000 sites together you’re well aware of all the pain caused by a small range of private AS numbers defined in RFC 1930. We can kludge our way around the limitation by reusing the same AS number on multiple sites (and using allowas-in when we need full routing information on every site), but such a design clearly sucks.

The next small step in my MPLS/VPN update project: EIBGP load balancing – why is it useful, how it works, why can’t you use it without full-blown MPLS/VPN, and what the alternatives are.

Andrew is struggling with MPLS/VPN providers and sent me the following question:

Is "carriers carrier" a real service? I'm having a bit of an issue at the moment with too many MPLS providers […] Carrier’s carrier would be an answer to many of them, but none of the carriers admit to being able to do this, so I was wondering if it's simply that I'm speaking to the wrong people, or whether they really don't...

Short answer: I have yet to see this particular unicorn roaming the meadows of reality.

Loads of niche features got crammed into (MP)BGP and MPLS since I wrote my MPLS books, most of them trying to tweak BGP (a scalable and reasonably slow routing protocol dealing with behemoth tables) to behave more like an IGP would.

It looks like we’ll never see updated versions of the books, so I’ll try to cover the new features with short videos. The first one on the list: BGP Best External – a mechanism that speeds up MP-IBGP convergence in primary/backup PE-CE scenarios using EBGP.

Arnold sent me an interesting challenge: he’s using two MPLS/VPN providers, with most sites being connected to both providers. He’d like to load balance the inter-site traffic across all PE-CE links – an easy task if you’re using RIP, OSPF or EIGRP as the PE-CE routing protocol, but he happens to be using BGP.

Matthew Stone encountered another unintended consequence of full Internet routing in a VRF design: the TCAM on his 6500 was 80% utilized even though he has the new Sup modules with one million IPv4 routes.

A closer look revealed the first clue: L3 forwarding resources on a Cat6500 are shared between IPv4 routes and MPLS labels (don’t know about you, but I was not aware of that) and half the entries were consumed by MPLS labels:

One of my readers sent me a surprising question: “We run only LDP in our MPLS network and need to run RSVP for TE and then phase out LDP. How could we do it?”

My first reaction was “Why would you ever want to do that” and I got no reasonable answer (suggestions, anyone?) but let’s focus on “Could you do it?”

TL&DR summary: You could, but that doesn’t mean you should.