Category: MPLS VPN
The Big Picture and my webinars (with a VPLS example)
Ever since I’ve figured out how to explain complex topics to bright engineers, I wanted to develop content (books, courses, documents) that explained (in this order):
- The Big Picture and WIIFM (What will the student gain by understanding and deploying something based on what I’m describing).
- How the technology we’re using actually works (remember: knowledge, not recipes) and finally
- How to configure, monitor and troubleshoot the actual boxes used to build the solution.
I’m positive you agree this approach makes perfect sense, and every now and then I’ve managed to get it right (for example, in the MPLS VPN books). Unfortunately, you’re often facing an uphill battle, as people want to focus on hands-on topics and hate to learn why things work the way they do instead of memorizing recipes like “Thou shalt not have more than 3 OSPF areas per router”.
Is anyone using MPLS/VPN over mGRE?
RFC 4364 (also known as RFC 2547bis from its draft days) specifies two methods of transporting VPN packets between PE-routers: well-known MPLS transport and GRE transport. The GRE transport idea is extremely simple: you take the labeled VPNv4 packet, put it into a GRE envelope, set the GRE Ethertype field to MPLS Unicast (0x8847) and send the GRE packet to the IP address of the egress PE-router.
This is not the same mechanism as running MPLS over point-to-point LDP-enabled GRE tunnels or running MPLS over DMVPN tunnels; this one does not require multiple tunnels or LDP/NHRP to work.
L2TP: The revenge of hardware switching
Do you like the solutions to the L2TP default routing problem? If you do, the ASR 1000 definitely doesn’t share your opinion; so far it’s impossible to configure a working combination of L2TP, IPSec (described in the original post) and PBR or VRFs:
PBR on virtual templates: doesn’t work.
Virtual template interface in a VRF: IPSec termination in a VRF doesn’t work.
L2TP interface in a VRF: This one was closest to working. In some software releases IPSec started, but the L2TP code was not (fully?) VRF-aware, so the LNS-to-LAC packets used global routing table. In other software releases IPSec would not start.
L2TP default routing: solutions
There are three tools that can (according to a CCIE friend of mine) solve any networking-related problems: GRE tunnels, PBR and VRFs. The solutions to the L2TP default routing challenge nicely prove this hypothesis; most of them use at least one of those tools.
Policy-based routing on virtual template interface. Use the default route toward the Internet and configure PBR with set default next-hop on the virtual template interface. The PBR is inherited by all virtual access interfaces, ensuring that the traffic from remote sites always passes the network core (and the firewall, if needed).
QPPB in MPLS VPN
TL&DR: QPPB works in MPLS VPNs… with a few limitations (at least in Cisco IOS implementation).
And now for the long story: A while ago I’ve noticed that my LinkedIn friend Joe Cozzupoli changed his status to something like “trying to get QPPB to work in MPLS VPN environment”. I immediately got in touch with him and he was kind enough to send me working configurations; not just for the basic setup, but also for Inter-AS Option A, B and C labs.
Knowing that QPPB relies on CEF, I doubted it would work as well on VRF interfaces as it does in pure IP environments, so I decided to do a few tests of my own. Here are the limitations I found:
Encrypting P-to-P-router traffic
Rob sent me a really good question:
I have an enterprise MPLS network. Two P routers are connected via carrier point-to-point Gigabit Ethernet and I would like to encrypt the MPLS traffic traversing the GE link. The PE-routers don't have hardware crypto accelerators, so I would like to keep the MPLS within the buildings running in cleartext and only encrypt the inter-site (P-to-P) MPLS traffic.
The only solution I could imagine would nicely fit the motto of one of our engineers: »Any time you have a problem, use more GRE tunnels« (if you have a better solution, please post it in the comments).
PE-to-PE IPSec: do you have creative ideas?
Ying would like to have a PE-to-PE IPSec protection for traffic within a single VRF. For example, all traffic in VRF-A sent between PE-1 and PE-2 should be protected with IPSec and the PE-routers should be the endpoints of the IPSec session (CE-to-CE IPSec is trivial).
My first response was “hard to do”, then I started hallucinating about MPLS-over-GRE-over-IPSec-over-IP-over-MPLS tunnels between the PE-routers with tunnel-specific IGP and per-VRF BGP next hops. It can be done (we’ve implemented numerous large-scale MPLS/GRE/IPSec designs), but is there a simpler alternative? Please share your ideas in the comments.
… updated on Tuesday, December 29, 2020 09:34 UTC
Limitations of VRF Routing Protocols on Cisco IOS
Cisco IOS allows up to 32 routing protocols contributing routes into a routing table (two of them are always connected and static). The limitation applies to the global routing table as well as to each individual VRF; the architectural reason for the limit is a 32-bit mask that’s used in Cisco IOS to mark individual routing protocols. The routing protocol ID (as displayed by the show ip protocol summary command) is thus limited to values 0 to 31. With value 0 being reserved for connected routes and value 1 for static routes, 30 values are left to number the routing protocols.
Due to the implementation details of Cisco IOS, the BGP, RIP and each EIGRP routing process consume routing protocol ID in all VRFs (regardless of whether they are used or not). You can view the IDs of individual routing protocols with the show ip protocol [vrf name] summary command.
True or false: MPLS VPNs offer equivalent security to Frame Relay
Years ago when vendors were pushing the MPLS story to Service Providers, an “independent analyst” wrote a report claiming that MPLS-based VPNs offer security equivalent to Frame Relay networks (to find those reports, ask Google about “MPLS security equivalent to Frame Relay”). This might be true from the functional perspective (and it’s absolutely true that using IP does not make MPLS-based VPNs inherently insecure), but anyone believing these reports might become mightily upset when learning about BGP and MPLS security issues.
Before going any further, please note that exploiting the MPLS architecture as described in All your packets are belong to us presentation requires access to the core SP network, which means that the network (or network management station) has already been successfully penetrated.
I hope you knew this already: Backbone-hacking tools
A few days ago (the) Steve Bellowin sent an e-mail to the NANOG mailing list pointing to a FUD-full article describing upcoming release of MPLS hacking tools. Christian Koch quickly pointed out a similar presentation given by the same group @ Schmoocon and numerous respondents correctly stated these are old issues … if you’re interested in BGP and MPLS security, of course. Nicolas Fischbach even provided link to a 7 year old presentation describing numerous BGP/IGP/MPLS risks and attack vectors.
Build a VPN Across Your IP Network with Multi-VRF Feature
One of our customers had to provide end-to-end IP transport across their enterprise network for an outsourced video surveillance solution. We implemented a true VPN solution for them (the hosts in the enterprise network cannot access the surveillance equipment and vice versa) using the Multi-VRF feature available in all recent Cisco IOS releases.
I described the solution Add a VPN to an Enterprise Network with Multi-VRF Functionality article; you’ll find it somewhere in this list.
Extranet with Overlapping Addresses
The idea to write an article describing how you can use MPLS VPN-enabled NAT to implement flexible extranets that allow participants to retain their existing (and sometimes overlapping) IP address space has been sitting in my to-do list for over a year.
After I’ve finally written it (without even hinting what I’ve been working on), I got several e-mails from my readers asking the questions this article answers, so it looks like the topic has suddenly become very hot. Do you have any ideas why that would be the case?
You’ll find the original article somewhere in this list.
MPLS support on 1800-series routers
Christoph sent me an interesting question a few days ago:
I played a bit arround with 2 Cisco 1803 and I found MPLS related configurations commands in IOS 12.4(15)T (Advanced Enterprise) on this box. MPLS was not listed as a included fearture in the Cisco Feature Navigator for this image and some searching at cisco.com took me to a 2 year old document telling me that MPLS isn't supported on this series. Some more searching took me back to the Cisco Feature Navigator which lists MPLS as feature for the Cisco 1805 router (which uses the same IOS image, afaik).
So, I'm a bit confused now if MPLS is really working / supported on the low-end Cisco ISR 1800 fixed series?
MPLS was mostly available but never supported on low-end platforms (including Cisco 2600). In those days I've taken some heat for reusing existing 2600-based labs to teach Cisco-internal MPLS courses (since we were teaching the students to configure unsupported devices :).
Anyhow, the "not supported" means exactly that: it may be available (well, it is), it may work (it actually does), but if it's broken (and I've seen at least one low-end-platform-specific bug in the early days) you can't complain.
Is anyone aware whether the official support for the MPLS on 1800 series has changed? If so, please share your information with us.
If you need to offer a production-grade service to your customers, don't use unsupported equipment; if you need a solution for your personal needs or you're building a lab, go ahead.
An offer you should not refuse
, MPLS and VPN Architectures, Volume II
and Internet Routing Architectures (2nd Edition) (Networking Technology)
(from Sam Halabi) for a total of $160.
Do you need LDP with MPLS TE?
An anonymous commenter to my implicit NULL/PHP post made a very valid point:
Most Cisco documentation states that you must enable LDP before doing MPLS-TE, which is a complete fallacy.
If you're using MPLS TE simply to shift IP traffic around your network, he's absolutely right: there is no need to run LDP if you have an IP-only network. If you're running MPLS VPN or BGP on edges/MPLS in the core, the answer becomes “it depends.”
I documented the detailed rules and undesired side effects if you ignore them a long while ago, but that article disappeared into /dev/null. Fortunately archive.org caught a copy before that.
Would you like me to migrate that article to ipSpace.net? Send me a message and I just might do it...