Is anyone using MPLS/VPN over mGRE?

RFC 4364 (also known as RFC 2547bis from its draft days) specifies two methods of transporting VPN packets between PE-routers: well-known MPLS transport and GRE transport. The GRE transport idea is extremely simple: you take the labeled VPNv4 packet, put it into a GRE envelope, set the GRE Ethertype field to MPLS Unicast (0x8847) and send the GRE packet to the IP address of the egress PE-router.

This is not the same mechanism as running MPLS over point-to-point LDP-enabled GRE tunnels or running MPLS over DMVPN tunnels; this one does not require multiple tunnels or LDP/NHRP to work.

The implementation of this simple mechanism in Cisco IOS release 12.0S would make Rube Goldberg envious, but I’ve finally got it up and running in my lab (and I understand most of what’s going on behind the scenes). 12.2 SRE hides almost all the complexity (although the implementation details remain mostly unchanged), but its configuration is totally incompatible with the previous implementation.

If there’s anyone truly interested in this mechanism, I’ll try to find some time to document it (Cisco’s documentation is “somewhat” misleading), otherwise I’ll just archive it together with a few other “interesting” IOS features I’ve tested (and decided to forget as soon as possible).

Recent posts in the same categories

22 comments:

  1. Joe Cozzupoli 15 April 2010 07:42
    Ivan,

    I'm definitely interested in testing and playing with this feature. Please document it on your blog or email it to me, you have my details.

    Cheers,
    Joe.
  2. Petr Lapukhov 15 April 2010 08:04
    It seems to me you are talking about BGP VPNs over mGRE, which Cisco implements using Resolve in VRF (RIV) "hack" (kind of neat idea, though). Obviously, BGP VPNs over P2P GRE tunnels are pretty trivial, and not as fun as the mGRE counterpart. I believe It would probably worth blogging about Tunnel SAFI and MP-BGP VPNs over L2TPV3 as well.
  3. Ivan Pepelnjak 15 April 2010 08:25
    You’re right: it’s the RIV hack (although it’s well hidden in 12.2 release) and MPLS VPN over P2P GRE tunnel is plain boring (and it brings you back to PVC world).
    Thanks for the L2TPv3 idea ... I knew I could count on creative ideas coming from you ;)
  4. chris 16 April 2010 16:01
    If you're interested in the L2TPv3, I wrote quickly something about this:

    http://www.ipflow.utc.fr/index.php/MPLS/VPN_over_L2TPv3_multipoint_tunnels
  5. Ivan Pepelnjak 16 April 2010 20:21
    Thank you for the link. It's almost exactly the same configuration as what I did with mGRE (12.0S, I presume), just using l2tpv3 instead of gre.
  6. chris 17 April 2010 09:40
    Hmm I feel very stupid now :) I didn't think that I could use another type of tunnel, I took the entire thing as a whole feature... Indeed that was with 12.0(33)S. I'm interested in the 12.2SR config if it is completely different from 12.0S.
  7. Ivan Pepelnjak 17 April 2010 19:21
    Sure, no problem. Just let me know where you want me to send it ;)
  8. ODIN 20 April 2010 07:44
    There is another problem with MPLS VPN and GRE: it is not possible to use TE. RSVP doesnt't work on GRE tunnel interfaces (heh... I want Juniper =)). Actually we have many CE-routers (2811) and we are dreaming about our own "virtual" MPLS backbone with TE. Who knows any way to build this? (SIG! only 2811)
  9. Ivan Pepelnjak 20 April 2010 10:09
    I have a vague feeling we’ve got MPLS TE working on a GRE tunnel once. Need to check with the engineer who configured that particular network.
    Anyhow, why would you want to build MPLS TE network over GRE tunnels? MPLS VPN I can understand (and we have multiple customers doing that).
  10. ODIN (Petr Ankudinov) 20 April 2010 13:33
    All this CE-routers are already connected through MPLS VPN cloud. But this cloud belongs to our ISPs. The idea is to virtualize our BGP-core. Then it would be easier to control routing tables, it would be possible to use route-targets like some kind of access control between our remote poins and more. But it is not interesting enough to redesign network without TE (to use FRR, control load, etc.). If RSVP will work, I think it can be valuable.
  11. ODIN (Petr Ankudinov) 20 April 2010 13:35
    All this CE-routers are already connected through MPLS VPN cloud. But this cloud belongs to our ISPs. The idea is to virtualize our BGP-core. Then it would be easier to control routing tables, it would be possible to use route-targets like some kind of access control between our remote poins and more. But it is not interesting enough to redesign network without TE (to use FRR, control load, etc.). If RSVP will work, I think it can be valuable.
  12. ODIN (Petr Ankudinov) 20 April 2010 14:12
    In short: main target is fast, but scalable core.
  13. Wiggo 22 April 2010 10:15
    Anyone know if this solution will support more than one label(other services than vpn)? If I use route reflectors in this design, do they also have to have the MPLS o mgre support? As for now it seems it is only supported on 7600 with sip 400, asr and isr g2 is coming later this year.

    Cheers,
    Wiggo.
  14. Ivan Pepelnjak 22 April 2010 16:21
    While the functionality seems to be purely MPLS/VPN-related, it could potentially support CsC. A test would be in order to support this speculation ;)

    Route reflectors work well with the 12.2 syntax; they just need to support VPNv4 AF. 12.0S is trickier.

    The feature runs on a 7200. 7200 is the only supported platform for 12.0S release; it might not be supported with 12.2SRE. You obviously know more about ASR and ISR than I do.
  15. dbm 23 April 2010 13:49
    There is actually a pair of documents from cisco, they are cisco validated designs 1 and 2.
    They are described as "next generation wan architecture" with mpls.
    One of the chapters in the CVDs is called 2547oDMVPN. As the title describes, cisco has tested (and now deployed on at least one large customer network) building mpls VPNs over DMVPN environments. It includes bugs, caveates, some config, etc. Its worth a read if you are looking into CSC or deploying MPLS for the enterprise.
  16. Ivan Pepelnjak 23 April 2010 16:50
    Yeah, I'm aware of those documents; we've been discussing MPLS-over-DMVPN in this blog a while ago. This one is almost identical from the encapsulation perspective, but uses completely different configuration.
  17. benner 16 May 2011 22:40
    Ivan,

    currently I am looking in to some options to extend a customer MPLS network... As far as I can see, I have to open options: DMVPN and mGRE... Not that I dont want to do deep investigation on both, but maybe you can easily give some pro's and cons about each of the solutions. I am still looking high level and for the moment it is not yet clear to me which solution is best.
  18. Ivan Pepelnjak 17 May 2011 16:31
    Do you mean DMVPN versus MPLS-over-mGRE (the real stuff where you don't have to configure the tunnels?). I would always go with MPLS-over-mGRE if you can keep the mGRE part in a separate AS.

    Also keep in mind that IPsec is an integral part of DMVPN whereas you might still need an encryption solution with mGRE (GETVPN would be the most scalable option).

    If you need in-depth design recommendation/review, consider this: http://www.ioshints.info/ExpertExpress
  19. Ben Blendeman 26 May 2011 18:34
    Ivan,

    thanks for your reply... indeed GETVPN should be an option to have MPLSomGRE encrypted and I was already looking in to that, but can not find a thing about those technologies mixed... The question is if this is a good solution in real life (where QoS and multicast needs to be supported). Are there any references out there in the field?

    Ben
  20. Ivan Pepelnjak 26 May 2011 20:16
    Great architecture, meet harsh reality ;) Will write a blog post about potential caveats.

    Don't have any GETVPN-related references (but maybe someone else has). We do run MPLS over GRE over P2P IPsec tunnels and it works on a large variety of platforms.
  21. kwame 09 November 2011 20:19
    Yes we're running MPLS VPN over mGRE on our production network. It is the core technology for upgrading our WAN, close to 600 sites, from ATM and FR. We've upgraded just about some 30 sites and we've hit a snag. As suggested by Ivan, we've a 2-AS design with the mGRE in the WAN and the non-mGRE in the Campus. Our WAN provider is ATT. Inside of the WAN things work like a charm. However VRF traffic between the 2 regions is a problem. We are dropping packets in either direction and it's limited to just VRF traffic. The ASBRs double as RRs for their respective AS. The WAS ASBR is a c7606 and the Campus ASBR is an ASR1002. We're now at a standstill trying to figure out the problem. By the way we connect to ATT over OC12 so it's not bandwidth constrained yet. Is anyone experiencing anything like this? We opted for this solution given its simplicity relative to the DMPVN solution. Had lots of help from Petr Lapukhov of INE in figuring the DMVPN solution out especially with regards to multicast.
  22. Ivan Pepelnjak 09 November 2011 22:22
    Based on your description, it seems the c7600 might do CPU-based forwarding (MPLSoGRE is a problem), which absolutely kills it.
Add comment
Home
Sidebar