Category: MPLS VPN

Loads of niche features got crammed into (MP)BGP and MPLS since I wrote my MPLS books, most of them trying to tweak BGP (a scalable and reasonably slow routing protocol dealing with behemoth tables) to behave more like an IGP would.

It looks like we’ll never see updated versions of the books, so I’ll try to cover the new features with short videos. The first one on the list: BGP Best External – a mechanism that speeds up MP-IBGP convergence in primary/backup PE-CE scenarios using EBGP.

Arnold sent me an interesting challenge: he’s using two MPLS/VPN providers, with most sites being connected to both providers. He’d like to load balance the inter-site traffic across all PE-CE links – an easy task if you’re using RIP, OSPF or EIGRP as the PE-CE routing protocol, but he happens to be using BGP.

Matthew Stone encountered another unintended consequence of full Internet routing in a VRF design: the TCAM on his 6500 was 80% utilized even though he has the new Sup modules with one million IPv4 routes.

A closer look revealed the first clue: L3 forwarding resources on a Cat6500 are shared between IPv4 routes and MPLS labels (I don’t know about you, but I was not aware of that), and half the entries were consumed by MPLS labels:

One of my readers sent me a surprising question: “We run only LDP in our MPLS network and need to run RSVP for TE and then phase out LDP. How could we do it?”

My first reaction was “Why would you ever want to do that” and I got no reasonable answer (suggestions, anyone?) but let’s focus on “Could you do it?”

TL&DR summary: You could, but that doesn’t mean you should.

Erich has encountered a familiar MPLS/VPN design challenge:

We have Cisco's 2901s with the data license running MPLS/VPN on customer site (the classical PE is at the customer site). Should we use eBGP between CPE router and network edge router, some sort of iBGP route reflector design, or something completely different?

The “it depends” answer depends primarily on how much you can trust the routers installed at the customer site (CPE routers).

One of my readers sent me an interesting problem a few days ago: the BGP process running on a PE-router in his MPLS/VPN network preferred an iBGP route received from another PE-router to a locally sourced (but otherwise identical) route. When I looked at the detailed printout, I spotted something “interesting” – the pre-bestpath cost extended BGP community.

During the February Packet Party, someone asked the evergreen question: “Is it safe to run Internet services in a VRF?” and my off-the-cuff answer was (as always) “Doing that will definitely consume more memory than having the Internet routes in the global routing table.” After a few moments, Derick Winkworth looked into one of his routers and confirmed the difference is huge ... but then he has a very special setup, so I decided to do a somewhat controlled test.

Whenever I’m explaining MPLS/VPN technology, I recommend using the same route targets (RT) and route distinguishers (RD) in all VRFs belonging to the same simple VPN. The Single RD per VPN recommendation doesn’t work well for multi-homed sites, so one might wonder whether it would be better to use a different RD in every VRF. The RD-per-VRF design also works, but results in significantly increased memory usage on PE-routers.

A lot of engineers are concerned with what seems to be frivolous creation of new encapsulation formats supporting virtual networks. While STT makes technical sense (it allows soft switches to use existing NIC TCP offload functionality), it’s harder to figure out the benefits of VXLAN and NVGRE. Scott Lowe wrote a great blog post recently where he asked a very valid question: “Couldn’t we use MPLS over GRE or IP?” We could, but we wouldn’t gain anything by doing that.

A while ago I wrote that the hypervisor vendors should consider turning the virtual switches into PE-routers. We all know that’s never going to happen due to religious objections from everyone who thinks VLANs are the greatest thing ever invented and MP-BGP is pure evil, but there are at least two good technical reasons why putting MPLS/VPN (as we know it today) in the hypervisors might not be the best idea in very large data centers.