MPLS/VPN Carrier’s Carrier – Myth or Reality?
Andrew is struggling with MPLS/VPN providers and sent me the following question:
Is "carriers carrier" a real service? I'm having a bit of an issue at the moment with too many MPLS providers […] Carrier’s carrier would be an answer to many of them, but none of the carriers admit to being able to do this, so I was wondering if it's simply that I'm speaking to the wrong people, or whether they really don't...
Short answer: I have yet to see this particular unicorn roaming the meadows of reality.
Carrier’s Carrier (CsC) seemed like a perfect idea for wholesale (transit) provides in the early days of MPLS – I know at least one of the big ones was pushing very hard to get it done. Transit providers could use CsC to offer their MPLS/VPN services as a transit MPLS backbone for other (smaller) MPLS/VPN providers.
However, even though CsC it was conceptually simple (all you had to do was to configure LDP or EBGP with labels on PE-CE link) I’ve never seen it working, and I’m guessing that most of the transit providers prefer offering VPLS (or other EoX) services – they’re way less hassle for the same money.
On the other hand, Carrier’s Carrier is nothing more than the ability to build your own MPLS/VPN service on top of someone else’s infrastructure, and probably won't solve your inter-provider issues. If you’re experiencing routing problems with your MPLS/VPN providers, stop bothering – use directly-connected subnets as transport endpoints, build GRE or MGRE tunnels on top of the providers’ MPLS/VPN network, and run your own routing protocols on top of them.
:-)
Kieran
I had the same mixed feelings about IP Multicast in the CCIE R&S exam.
That means if given ISP would have an option to plan the network from the very start (taking acqusitions into account), they would be more likely going for another architecture.
But I believe such implementation must exist in large countries like US and Russia, where even large MPLS service providers cannot cover whole country with private optics, and are forced to lease lambdas or use other SPs VPN services.
Certainly, layer2 VPNs will be much simpler solution for this situation, but again, for large SPs it can be an awful lot of point-to-point L2 VPNs, while VPLS will have a lot of drawbacks, so L3 MPLS VPN+labeled BGP with Carrier SP would perform best.
From a network operations view, L3VPNs in general are much easier for the SP NOC to troubleshoot with the customer since the CE and PE do exchange some NLRI which can be viewed by both the SP and the Customer NOCs. Most customers do not really have the ability to troubleshoot Ethernet services due to both lack of knowledge and tools (and the belief that Ethernet just magically works). This situation could be improved if both customers and service providers used CFM as a diagnostic tool, but I have yet to see this in the real world either.
Having said that, I will add that it is being deployed in atleast one less-than-public network - with mostly successful results thus far.
1. The POPs will have some hefty multicast flow requirements between one another, making VPLS a poor option for the core carrier.
2. CSC is very easy from a conceptual perspective. Its actually easier for us to implement CSC than if the carrier provided us a L2VPN. If they did that, I would either have to create a new IGP process for inter-POP communications, or use area 0 in across the world with non-area-0 in each POP (something like that). Then, I would have to either use seamless MPLS or leak PE loopbacks across IGP boundaries ... note that seamless MPLS and CSC are almost identical designs, except the former is for an intra-AS design :)
3. Maximum isolation of failure domains. I can add a new POP later, run eBGP+Label with the core carrier, and there is no significant change to the other N-1 POPs notwithstanding the BGP routes they'll learn from the newly-added POP. With VPLS, assuming I extended IGP across the globe, that changes a bit. Customer also has a stiff requirement for isolation since failures in a POP should minimally affect other POPs.
4. Label filtering is a breeze. I only want to allocate BGP labels for my PE loopbacks, while aggregating the networks of my POP transit links into something coarse (like a /16) so I can still have full IP reachability to assist my operations staff with troubleshooting. I suppose this is also true for LDP, but I only need to manage it at a single exchange point, rather than a big broadcast segment with many peers.
Moral of the story is that just because a service is not commonly offered does not mean it lacks value. Likewise, just because something is popular/hyped-up does not mean it has value. Thanks Ivan for the great conversation :)
-Nick