End-to-end QoS marking in MPLS/VPN-over-DMVPN networks

I got a great question in one of my Enterprise MPLS/VPN Deployment webinars (register here) when I was describing how you could run MPLS/VPN across DMVPN cloud:

That sounds great, but how does end-to-end QoS work when you run IP-over-MPLS-over-GRE-over-IPSec-over-IP?

My initial off-the-cuff answer was:

Well, when the IP packet arriving through a VRF interface gets its MPLS label, the IP precedence bits from the IP packet are copied into the MPLS EXP (now TC) bits. As for what happens when the MPLS packet gets encapsulated in a GRE packet and when the GRE packet is encrypted ... I have no clue. I need to test it.

Finally I found some time to do proper lab tests. I fired up one of my MPLS/VPN-over-DMVPN labs, turned one of the remote site routers (R1A in the following diagram) into a traffic source host (by disabling all its interface but its LAN interface) and started my tests.

Have I told you that you get 10 different sets of router configurations, covering everything from Multi-VRF to MPLS/VPN-over-DMVPN with the Enterprise MPLS/VPN Deployment webinar? DMVPN: From Basics to Scalable Networks has even more labs: you get complete router configurations for more than 20 full-blown DMVPN labs, each one having 2 hub routers and 4 spoke routers.

MPLS label imposition. The default behavior of Cisco IOS is to copy IP precedence bits (the upper part of DSCP value) into MPLS EXP (aka TC) bits during label imposition.

Update 2011-02-05: This part of the post has been totally rewritten as the unusual behavior I “discovered” in my early lab tests turned out to be totally irreproducible.

You can change the default behavior by specifying the desired value in a policy-map that you apply as inbound service-policy on the VRF interface (VPN label imposition seems to be conceptually tied to the incoming VRF interface, not the outgoing MPLS interface). For example, to ignore the IP precedence bits on untrusted interfaces and set MPLS EXP/TC bits to zero, use the following QoS service policy:

policy-map SetMPLSExp
 class class-default
  set mpls experimental imposition 0
!
interface FastEthernet0/0
 ip vrf forwarding VPN
 ip address 172.16.10.2 255.255.255.0
 service-policy input SetMPLSExp

The need to configure inbound service policy surprised me (I wanted to configure outbound service policy on the DMVPN interface), but is actually a great solution: you can apply different policies to different VRF interfaces and trust IP precedence markings on per-interface basis.

Propagation of MPLS EXP bits into GRE/IPSec headers. I was pleasantly surprised to find out that the MPLS EXP bits get copied into IP precedence bits of the GRE envelope with no extra configuration. As I’m always using IPSec in transport mode in DMVPN designs, IPSec encryption does not touch the IP header (including IP precedence) and thus we get the desired end-to-end IP precedence markings with minimal efforts.

9 comments:

  1. as far as i can remember, the default behavior when imposing labels is indeed to reflect the precedence to exp. On the eLER the default will be to remove the label stack WITHOUT copying the exp back to precedence - hence, Cisco IOS use the (short) pipe model.

    But maybe they changed this default some place along the IOS way

    my 2 cent

    ReplyDelete
  2. Yep, only TTL is copied from IP header to MPLS header at the ingress LSR (subject to fading memory :-P ).

    ReplyDelete
  3. So nice to hear I'm not the only one remembering things past ;)

    ReplyDelete
  4. Interesting Ivan, however I found the exact opposite when checking MPLS QoS labels... http://davidrothera.co.uk/cisco/2011/02/mpls-qos-marking-the-rebuttal/

    ReplyDelete
  5. Thank you for the feedback. I was using 15.0M, so it looks like the change in behavior was pretty recent.

    ReplyDelete
  6. Update: Looks like I was doing something wrong during my lab tests. I repeated the tests today and MPLS/VPN-over-DMVPN works as we all expected. Reworded the post.

    ReplyDelete
  7. So we had "IPv4/v6 over MPLS over GRE over IPSec over XXX"
    together with Short Pipe since 12.4T. it's working.
    with 15T we have discovered a "NEW" mapping feature
    "Marking Copy from GRE to IPSec".
    In our case it is very good in order to signal QoS-Marking into
    transmissions networks. (e.g. 4SLA)...

    ReplyDelete
  8. Hasn't the GRE-to-IPsec ToS copying worked "forever"? Tunnel-mode IPsec might be an issue, but transport-mode IPsec should work out-of-the-box.

    ReplyDelete
  9. we used IPSec default (tunnel-mode) and now "mode transport require"
    if possible...
    BTW a little challenge was also ISIS in this QoS context -> IPSec ToS = 0.

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.