Category: MPLS VPN

A while ago John McManus wrote a great DSCP QoS Over MPLS Thoughts article at Etherealmind blog explaining how 6-bit IP DSCP value gets mapped into 3-bit MPLS EXP bits (now renamed to Traffic Class field). The most important lesson from his post should be “there is no direct DSCP-to-EXP mapping and you have to coordinate your ideas with the SP”. Let’s dig deeper into the SP architecture to truly understand the complexities of this topic.

We’ll start with a reference diagram: user traffic is flowing from Site-A to Site-B and the Service Provider is offering MPLS/VPN service between PE-A and PE-B. Traffic from multiple customer sites (including Site-A) is concentrated at SW-A and passed in individual VLANs to PE-A.

A while ago Ron sent me an intriguing question: “Is it possible to have two EIGRP AS numbers in the same VRF?” Obviously he’s working on a network with multiple EIGRP processes (not an uncommon pre-MPLS/VPN solution; I did a network design along the same lines almost 20 years ago).

It’s easy to run multiple EIGRP autonomous systems in the global IP routing table; just create more than one EIGRP process. They can even run over the same set of interfaces. EIGRP-in-a-VRF implementation is slightly different; you configure an address family within another EIGRP process and (optionally) specify an AS number that does not have to match the AS number of the EIGRP process.

Ever since I’ve figured out how to explain complex topics to bright engineers, I wanted to develop content (books, courses, documents) that explained (in this order):

• The Big Picture and WIIFM (What will the student gain by understanding and deploying something based on what I’m describing).
• How the technology we’re using actually works (remember: knowledge, not recipes) and finally
• How to configure, monitor and troubleshoot the actual boxes used to build the solution.

I’m positive you agree this approach makes perfect sense, and every now and then I’ve managed to get it right (for example, in the MPLS VPN books). Unfortunately, you’re often facing an uphill battle, as people want to focus on hands-on topics and hate to learn why things work the way they do instead of memorizing recipes like “Thou shalt not have more than 3 OSPF areas per router”.

RFC 4364 (also known as RFC 2547bis from its draft days) specifies two methods of transporting VPN packets between PE-routers: well-known MPLS transport and GRE transport. The GRE transport idea is extremely simple: you take the labeled VPNv4 packet, put it into a GRE envelope, set the GRE Ethertype field to MPLS Unicast (0x8847) and send the GRE packet to the IP address of the egress PE-router.

This is not the same mechanism as running MPLS over point-to-point LDP-enabled GRE tunnels or running MPLS over DMVPN tunnels; this one does not require multiple tunnels or LDP/NHRP to work.

Do you like the solutions to the L2TP default routing problem? If you do, the ASR 1000 definitely doesn’t share your opinion; so far it’s impossible to configure a working combination of L2TP, IPSec (described in the original post) and PBR or VRFs:

PBR on virtual templates: doesn’t work.

Virtual template interface in a VRF: IPSec termination in a VRF doesn’t work.

L2TP interface in a VRF: This one was closest to working. In some software releases IPSec started, but the L2TP code was not (fully?) VRF-aware, so the LNS-to-LAC packets used global routing table. In other software releases IPSec would not start.

There are three tools that can (according to a CCIE friend of mine) solve any networking-related problems: GRE tunnels, PBR and VRFs. The solutions to the L2TP default routing challenge nicely prove this hypothesis; most of them use at least one of those tools.

Policy-based routing on virtual template interface. Use the default route toward the Internet and configure PBR with set default next-hop on the virtual template interface. The PBR is inherited by all virtual access interfaces, ensuring that the traffic from remote sites always passes the network core (and the firewall, if needed).

TL&DR: QPPB works in MPLS VPNs… with a few limitations (at least in Cisco IOS implementation).

And now for the long story: A while ago I’ve noticed that my LinkedIn friend Joe Cozzupoli changed his status to something like “trying to get QPPB to work in MPLS VPN environment”. I immediately got in touch with him and he was kind enough to send me working configurations; not just for the basic setup, but also for Inter-AS Option A, B and C labs.

Knowing that QPPB relies on CEF, I doubted it would work as well on VRF interfaces as it does in pure IP environments, so I decided to do a few tests of my own. Here are the limitations I found:

Rob sent me a really good question:

I have an enterprise MPLS network. Two P routers are connected via carrier point-to-point Gigabit Ethernet and I would like to encrypt the MPLS traffic traversing the GE link. The PE-routers don't have hardware crypto accelerators, so I would like to keep the MPLS within the buildings running in cleartext and only encrypt the inter-site (P-to-P) MPLS traffic.

The only solution I could imagine would nicely fit the motto of one of our engineers: »Any time you have a problem, use more GRE tunnels« (if you have a better solution, please post it in the comments).

Ying would like to have a PE-to-PE IPSec protection for traffic within a single VRF. For example, all traffic in VRF-A sent between PE-1 and PE-2 should be protected with IPSec and the PE-routers should be the endpoints of the IPSec session (CE-to-CE IPSec is trivial).

My first response was “hard to do”, then I started hallucinating about MPLS-over-GRE-over-IPSec-over-IP-over-MPLS tunnels between the PE-routers with tunnel-specific IGP and per-VRF BGP next hops. It can be done (we’ve implemented numerous large-scale MPLS/GRE/IPSec designs), but is there a simpler alternative? Please share your ideas in the comments.

Cisco IOS allows up to 32 routing protocols contributing routes into a routing table (two of them are always connected and static). The limitation applies to the global routing table as well as to each individual VRF; the architectural reason for the limit is a 32-bit mask that’s used in Cisco IOS to mark individual routing protocols. The routing protocol ID (as displayed by the show ip protocol summary command) is thus limited to values 0 to 31. With value 0 being reserved for connected routes and value 1 for static routes, 30 values are left to number the routing protocols.

Due to the implementation details of Cisco IOS, the BGP, RIP and each EIGRP routing process consume routing protocol ID in all VRFs (regardless of whether they are used or not). You can view the IDs of individual routing protocols with the show ip protocol [vrf name] summary command.