Unequal cost load-sharing

One of the most commonly asked load-sharing-related questions is "can I load-share traffic across unequal-cost links". In general, the answer is no. In order to load-share the traffic, you need more than one path to the destination and the only way to get multiple routes toward a destination in the IP routing table is to make them equal-cost (the only notable exception being EIGRP that supports unequal-cost load-sharing with the variance parameter).

There are, however, two cases where you can force unequal traffic split across equal-cost paths toward a destination: when using inter-AS BGP with the link bandwidth parameter and when using unequal-bandwidth traffic-engineering tunnels.

Note: You can read more about load sharing with MPLS TE in my IP Corner article Perfect Load-Balancing: How Close Can You Get?Due to the way MPLS TE autoroute is implemented in Cisco IOS, all tunnels toward the same destination appear as equal-cost paths, even when their TE bandwidths are not the same. For example, using a simple TE configuration ...

interface Tunnel0
ip unnumbered Loopback0
tunnel destination 172.16.0.21
tunnel mode mpls traffic-eng
tunnel mpls traffic-eng autoroute announce
tunnel mpls traffic-eng priority 7 7
tunnel mpls traffic-eng bandwidth 300
tunnel mpls traffic-eng path-option 1 explicit identifier 1
no routing dynamic
!
interface Tunnel1
ip unnumbered Loopback0
tunnel destination 172.16.0.21
tunnel mode mpls traffic-eng
tunnel mpls traffic-eng autoroute announce
tunnel mpls traffic-eng priority 7 7
tunnel mpls traffic-eng bandwidth 500
tunnel mpls traffic-eng path-option 1 explicit identifier 2
no routing dynamic
... you get two equal-cost paths in your IP routing table even though the tunnel mpls traffic-eng bandwidths are different:
a1#show ip route ospf
172.16.0.0 255.255.0.0 is variably subnetted, 6 subnets, 2 masks
O 172.16.0.21 255.255.255.255 [110/51] via 0.0.0.0, 00:11:06, Tunnel0
[110/51] via 0.0.0.0, 00:11:06, Tunnel1
O 172.16.0.22 255.255.255.255 [110/52] via 0.0.0.0, 00:11:06, Tunnel0
[110/52] via 0.0.0.0, 00:11:06, Tunnel1
When transferring the IP routing table into the CEF table, the router takes MPLS TE bandwidth in consideration, resulting in unequal traffic split proportional to the MPLS TE bandwidth:
a1#show ip cef 172.16.0.21 internal
172.16.0.21/32, version 55, epoch 1, per-destination sharing
0 packets, 0 bytes
tag information set
local tag: tunnel-head
via 0.0.0.0, Tunnel0, 0 dependencies
traffic share 3
next hop 0.0.0.0, Tunnel0
valid adjacency
tag rewrite with Tu0, point2point, tags imposed: {}
via 0.0.0.0, Tunnel1, 0 dependencies
traffic share 5
next hop 0.0.0.0, Tunnel1
valid adjacency
tag rewrite with Tu1, point2point, tags imposed: {}

0 packets, 0 bytes switched through the prefix
tmstats: external 0 packets, 0 bytes
internal 0 packets, 0 bytes
Load distribution: 0 1 0 1 0 1 0 1 0 1 0 1 1 1 1 1 (refcount 1)

Hash OK Interface Address Packets Tags imposed
1 Y Tunnel0 point2point 0 {}
2 Y Tunnel1 point2point 0 {}
3 Y Tunnel0 point2point 0 {}
4 Y Tunnel1 point2point 0 {}
5 Y Tunnel0 point2point 0 {}
6 Y Tunnel1 point2point 0 {}
7 Y Tunnel0 point2point 0 {}
8 Y Tunnel1 point2point 0 {}
9 Y Tunnel0 point2point 0 {}
10 Y Tunnel1 point2point 0 {}
11 Y Tunnel0 point2point 0 {}
12 Y Tunnel1 point2point 0 {}
13 Y Tunnel1 point2point 0 {}
14 Y Tunnel1 point2point 0 {}
15 Y Tunnel1 point2point 0 {}
16 Y Tunnel1 point2point 0 {}
Note: this article is part of You've asked for it series.

8 comments:

  1. This sounds like a great feature.
    Sorry for a newbie question but does ip cef take into account the bandwidth setting on an interface so that dissimilar speed interfaces can be proportionally load balanced? I'm thinking about load sharing between an adsl and a cable modem. my application makes many connections to the same destination address so it would be nice to efficiently load share based on a source port. Will OER work better in this case?

    ReplyDelete
  2. CEF (or any other load-sharing mechanism) does not consider interface bandwidth (the only exception being MPLS TE tunnels). To perform non-equal load split you need a different approach, for example, creative use of static routes.

    ReplyDelete
  3. Hi! Sorry for a very basic and stupid question... For the tunnel you set up, the next hop for the destinations are all 0.0.0.0. In my testing scenarios the destination address of the tunnel are shown. Do you have any idea why it is 0.0.0.0? Maybe I got something wrong in my TE setup... Thanks a lot!!!

    ReplyDelete
  4. I don't have the original lab setup handy any more, so I would have to recreate it from scratch. Just guessing: I was using unnumbered interfaces. Did you assign IP addresses to the tunnel interfaces? In any case, the tunnel interfaces are point-to-point, so the next hop is effectively ignored (it's important only on multi-access LAN/WAN interfaces).

    ReplyDelete
  5. Thanks a lot for your information. I am using unnumbered tunnel ifc as well. It seems that this does not affect the routing (for the reason you mentioned) but I was just a little bit concerned.. :-)

    ReplyDelete
  6. if i change bandwidth on tunnel
    i have change RSVP reservation
    but i have no changel IGP/CEF load-share count
    any ideas ? ;)

    cost changed only after "clear ip route XXXXXX"

    ReplyDelete
  7. Obviously IP routing entries and CEF entries only change when the IP routing table changes ... which is not triggered by tunnel RSVP reservation, just by actual change in topology. No idea what else you could do.

    ReplyDelete
  8. hi friends,

    ip cef accounting load-balance-hash hidden command will enable the counters of packets forwarded in to links (fifth column).

    any one know how to clear it ? :P

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.