DMVPN as a backup for MPLS/VPN

SK left a long comment to my More OSPF-over-DMVPN Questions post describing a scenario I find quite often in enterprise networks:

  • Primary connectivity is provided by an MPLS/VPN service provider;
  • Backup connectivity should use DMVPN;
  • OSPF is used as the routing protocol;
  • MPLS/VPN provider advertises inter-site routes as external OSPF routes, making it hard to properly design the backup connectivity.

If you’re familiar with the way MPLS/VPN handles OSPF-in-VRF, you’re probably already asking the question “how could the inter-site OSPF routes ever appear as E1/E2 routes?”

The reason is quite simple: some service providers deploy their own CE-routers (see the diagram below, this time drawn with an improved version of the diagramming tool):

Hub-------DMVPN--------------Spoke
 !                             !
HCE--SPCE--PE--MPLS--PE-SPCE—SCE

The MPLS/VPN SP deploying its own CE-routers commonly runs EBGP between PE- and CE-routers and deploys OSPF only between its CE-routers (SPCE) and customer’s CE-routers (HCE and SCE). In this setup, the OSPF-specific MPLS/VPN attributes carried in MP-BGP are lost on the PE-CE router boundary and cannot be used to recreate the original OSPF route when the IP prefix is redistributed from BGP into OSPF on the provider’s CE-router. All inter-site routes thus appear as E1 or E2 routes in OSPF.

If you decide to run OSPF across the DMVPN cloud, the inter-site routes received over DMVPN will appear as intra-area or inter-area OSPF routes, making them more preferred than the MPLS/VPN routes (which should provide primary connectivity). Unless you want to be like Tom Cruise or MacGyver (who would indubitably find a solution involving duct tape, NAT, PBR and GRE), you have only two options:

Run a third routing protocol over DMVPN and redistribute it into OSPF with cost worse than the MPLS/VPN E1/E2 routes. Obviously you need to redistribute per-site OSPF routes into both protocols and you don’t know what the SP CE-routers are doing, which makes this design a nice two-way redistribution challenge worthy of the CCIE lab.

Give up on OSPF and run EBGP everywhere. Run EBGP between your CE-routers and SP’s CE-routers as well as over DMVPN (making the MPLS/VPN routes directly comparable to DMVPN routes) and use local preference to implement the routing policy you want. Obviously you’d still need to run OSPF within each site and IBGP between all site transit routers.

Need help?

Our professional services team did quite a few WAN network design projects solving the MPLS/DMVPN integration problems; if you need me (or one of our experts) for a few day on-site network design/review workshop, get in touch ... and there’s always the ExpertExpress service for smaller-scale challenges.

4 comments:

  1. We use this exact scenario, but with eBGP for the routing protocol as you describe at the end. It works great except a few caveats:

    1) we have found that we must use some sort of marking scheme (community in our case) on the hub side to ensure that remote site routes learned over the DMVPN path are not advertised to the MPLS VPN service from the hub
    2) a local pref/as path prepend scheme is needed to ensure MPLS VPN path is used over DMVPN
    3) a community-based scheme to set local-preference at the hub based on a community attached to a route advertised from the remote site can be very useful when in a restrictive change environment, as inbound and outbound paths are controlled from the remote site only.

    8-)

    We have had some concerns with running out of private AS numbers, as we use a similar scheme for customer connectivity on our extranet, so we are considering a model where we use iBGP and set the next hop with a route-map to make iBGP behave more like eBGP and to avoid the need to run an IGP to the remote sites over DMVPN for NLRI information.

    ReplyDelete
  2. Jon Harald Bøvre25 August, 2011 18:56

    Almost exactly as our configurations involving DMVPN.
    Running eBGP, let remote side decide return path using local-preference outbound and signaling MED inbound (seen from spoke)
    Always AS65000. as-override not configured on Hub, as spokes will get default route from hub anyway.

    ReplyDelete
  3. I used to work at a large law firm. We used OSPF for our internal routing and BGP for MPLS VPN. The edge routers used redistribution of OSPF into BGP. Worked well for us.

    ReplyDelete
  4. It's manageable if MPLS/VPN OSPF routes appear as inter-area routes. Use multiple areas over DMVPN and increase cost of DMVPN tunnels.

    The problem becomes harder if the SP manages the CE-router and sends you MPLS/VPN routes as E2 OSPF routes.

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.