A while ago Tomasz Kacprzynski asked me whether I'd ever run RSVP over DMVPN. I hadn't - after all, you'd only need that in VoIP environments and I try to stay as far away from voice as possible.
In the meantime, Tomasz solved the problem (short summary: you have to turn Phase 3 DMVPN into Phase 2 DMVPN) and wrote a lengthy blog post describing the problem (RSVP split horizon rule) and his solution (including numerous debugging printouts). Definitely worth reading if there's a non-zero chance you'll have to get the two working together.
Have you ever tried to implement a large-scale DMVPN or MPLS/VPN network using BGP as the routing protocol? If you tried to stitch more than ~1000 sites together you’re well aware of all the pain caused by a small range of private AS numbers defined in RFC 1930. We can kludge our way around the limitation by reusing the same AS number on multiple sites (and using allowas-in when we need full routing information on every site), but such a design clearly sucks.
Two years ago I wrote the another Fermatish post: I described how NHRP behavior changed in DMVPN networks using NAT and claimed that it might be a huge problem, without ever explaining what the problem is.
Fabrice quickly identified the problem, but it seems the description was not explicit enough as I’m still getting queries about that post ... so here’s a step-by-step description of what’s going on.
NHRP-based interface state control is a fantastic feature that you can use for faster convergence of very large DMVPN networks (as explained in the DMVPN Designs webinar, you can also use it to solve some interesting backup scenarios). We tested it in a network with over 1000 spokes (using ASR1K as the hub router) using very short registration timeouts, and the CPU utilization of the NHRP process rarely exceeded a few percents.
Chris sent me an interesting question that I haven’t covered in any of my DMVPN webinars: “How would you migrate a part of a Phase-1 DMVPN network to a Phase-2 or Phase-3 network if you can only migrate one spoke site at a time? Can I just upgrade the spokes that need spoke-to-spoke connectivity?”
While it might be theoretically possible to have a mixed Phase-1/Phase-2 DMVPN tunnel (and I just might be able to get it to work in a lab), such a solution definitely violates the KISS principle.
My next live webinar will be based on the DMVPN design posts I wrote recently and a number of scenarios that landed in my Inbox during the last few months. I’ll try to help you decide which phase of the DMVPN technology to use, which routing protocol would be best for you, and how to optimize the routing protocol you select. We’ll also discuss interesting redundancy and primary/backup scenarios, including combinations of DMVPN, MPLS/VPN and 3G networks.
In the Redundant DMVPN Design, Part 1 I described the options you have when you want to connect non-redundant spokes to more than one hub. In this article, we’ll go a step further and design hub and spoke sites with multiple uplinks.
Public IP addressing
Fact: DMVPN tunnel endpoints have to use public IP addresses or the hub/spoke routers wouldn’t be able to send GRE/IPsec packets across the public backbone.
Most of the DMVPN-related questions I get are a variant of the “how many tunnels/hubs/interfaces/areas do I need for a redundant DMVPN design?” As always, the right answer is “it depends” (and I can always help you with your design if you’d like to get a second opinion), but here’s what I’ve learned so far.
Got the following question with an invalid return address, so I’m broadcasting the reply ;)
I am running a DMVPN network and recently got a requirement for spoke-to-spoke communication. We currently shape traffic on a per spoke basis on the hub, and have a single shaper at the remote site. However, if a spoke is receiving a large amount of traffic from the hub and another spoke site, how will the sites sending traffic know that the remote port is congested?
Short answer – they won’t. You have a mission-impossible problem (very similar to ADSL QoS), but there might be some slight silver lining:
One of my readers sent me the following question a few days ago:
Do you have a webinar that covers Dual DMVPN HUB deployment using OSPF? If so which webinar covers it?
I told him that the DMVPN: From Basics to Scalable Networks webinar covers exactly that scenario (and numerous others), describing both Phase 1 DMVPN and Phase 2 DMVPN design and implementation guidelines. Interestingly, he replied that the information on this topic seems to be very scant:
SK left a long comment to my More OSPF-over-DMVPN Questions post describing a scenario I find quite often in enterprise networks:
- Primary connectivity is provided by an MPLS/VPN service provider;
- Backup connectivity should use DMVPN;
- OSPF is used as the routing protocol;
- MPLS/VPN provider advertises inter-site routes as external OSPF routes, making it hard to properly design the backup connectivity.
If you’re familiar with the way MPLS/VPN handles OSPF-in-VRF, you’re probably already asking the question “how could the inter-site OSPF routes ever appear as E1/E2 routes?”
Warning: totally shameless plug ahead. You might want to stop reading right now.
Every now and then one of the engineers listening to my webinars shares a nice success story with me. One of them wrote:
I'm doing a DMVPN deployment and Cisco design docs just don’t cover dual ISPs for the spokes hence I thought I give your webinars/configs a try.
... and a bit later (after going through the configs that you get with the DMVPN webinar):
After weeks of waiting, perfect summer weather finally arrived ... and it’s awfully hard to write blog posts that make marginal sense when being dead-tired from day-long mountain biking, so I’ll just recap the conversation I had with Brian a few days ago. He asked “How would I set up a (dual) hub running OSPF with phase 1 spokes and prevent all spoke routes from being seen at other spokes? Think service provider environment.”
New DMVPN features in IOS release 15.x is obviously a topic without a broad audience ... although Cisco did introduce some nifty new things that can help you scale a large DMVPN network or make a DMVPN network more manageable.
Got this question a few days ago:
I have a large DMVPN network (~ 1000 sites) using variety of DSL, cable modem, and wireless connections. In all of these cases the bandwidth is extremely dissimilar and even varies with time. How can I handle this in a scalable way? Also, do you know of any product or facility that I can use to better measure the bandwidth from hub to spoke and better set the QOS values?
The last question is the easy part: one of the products that does that is NIL Monitor service where the remote probes can measure the actual end-to-end bandwidth. NIL Monitor software can also log into routers and change configurations if needed ... but what should you change?