Per-destination or per-packet CEF load sharing?

Cisco Express Forwarding (CEF) can perform per-packet or per-destination (actually source/destination IP address pair) load-sharing with no performance degradation (without CEF, per-packet load-sharing requires process switching). Even though there is no performance impact on the router, per-packet load sharing will almost always result in out-of-order packets. The packet reordering might degrade TCP throughput in high-speed environments (in low-speed/few-flows scenarios, per-packet load-sharing actually improves the per-flow throughput) or severely impact applications that cannot survive out-of-order packet delivery, such as Fast Sequenced Transport for SNA over IP or voice/video streams.

To configure per-packet load-sharing, use the ip load-sharing per-packet interface configuration command (default is per-destination). This command has to be configured on all outgoing interfaces over which the traffic is load-shared.

The switch between the load-sharing modes is not immediate; sometimes you have to wait a few seconds for the ip load-sharing command to take effect, worst case a manual clearing of the CEF table (clear ip cef address) is required.

10 comments:

  1. In a scenario where customer takes links from two service providers and wants to load share the traffic between the links, will equimetric default routes in the routing table (and thereby in the cef table) actually load share per packet or per destination or per session ?

    Ramya

    ReplyDelete
  2. The CEF load-sharing mechanism depends on what you've configured on outgoing interfaces (links toward the ISPs) with the ip load-sharing configuration command.

    Load sharing can be per-source-destination-pair (default), per-packet (strongly discouraged in this scenario, as the packets will be almost guaranteed to arrive out-of-order) or per-flow (see Per-port CEF load sharing).

    ReplyDelete
  3. Do you know if "show ip cef exact-route" command is supported for 3560 switches?. It seems that the output of the command shows not correct information.

    ReplyDelete
  4. No idea :( Plus I don't have any 3560 handy ...

    ReplyDelete
  5. Hi, Great site. With Per-destination or Per-Packet switching, how does both interact with a dynamic routing protocol like EIGRP which has variance or can consider multiple links and perform unequal-cost load balancing? Does the "switching" override the routing protocol?
    I'm a bit unclear about this?
    And especially if BGP is involved!
    Thanks.

    ReplyDelete
  6. The "switching" part takes place after the routing table has been built and cannot influence the routing table at all. If you're using EIGRP variance or BGP DMZ bandwidth, the ratio of entries in the CEF table reflects the ratio between the path costs.

    ReplyDelete
  7. How many link can be used for load sharing
    I have 4 running OK but was wondering if there was limit encase they want to add more?


    Wayne

    ReplyDelete
  8. The absolute maximum CEF can support is 16 (the hash values used to select outgoing interface are 4-bit long). With anything even close to that limit you'll not get perfect load sharing due to rounding errors.

    If you have significantly more than four parallel links, you might consider upgrading to a higher-speed solution.

    ReplyDelete
  9. hosein abdollahi24 May, 2010 19:48

    some interfaces did not support ip load-sharing per-packet . so is there any solutions for it O:-)

    ReplyDelete
  10. it dont works in XR if the interfaces are not in a bundle?

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.