Load balancing quirks

One of my readers has noted an interesting load-balancing behavior: when he was running traceroute tests from various routers in a topology similar to the one displayed below, the traceroute outputs indicated per-packet load balancing (both paths were used) when they were initiated from R2 or R3, but used a single path when initiated from R1 or R4.

 
The reason for this behavior is very simple: if you do traceroute from R1 to R4, R2 and R3 perform CEF switching, which usually does load balancing based on source-destination IP address pairs, so all probe packets from R1 to R4 travel along the same path. If you start traceroute from R2 or R3, the packets are process-switched on the first hop (from R2 to R3, for example) and thus alternate between the parallel links.

This article is part of You've asked for it series.

5 comments:

  1. Hi Ivan,

    Could you please elaborate a little more on this. I thought with CEF switching even the first packet was CEF switched. I believe that was one of the advantages of cef over fast switching. Thanks.

    ReplyDelete
  2. Regarding above comment:

    That's only when the packet is flowing through the router, and not originating or terminating on the router. In this case, R2 or R3 have to craft the actual traceroute packet, and in doing so its process switched -- and hence load-balanced across both links.

    ReplyDelete
  3. Nice post. For who's interested there is a good document CEF cef load balancing here
    http://www.cisco.com/en/US/products/hw/modules/ps2033/prod_technical_reference09186a00800afeb7.html

    ReplyDelete
  4. Hello if we modify the above diagram and instead of 2 links between R2 and R3 we take

    - 1 of the links and put it to another device say R2 to R5
    - R5 has another link to R4

    If you do a extended traceroute from R2 with probe count of 1 is this behavior normal/possible

    1.First probe packet with ttl 1 gets sent to R3 than the icmp-time-exceeded reply comes from R3 hence the R3 ip address gets recorded as the first path

    2. R2 sends another probe with ttl 2 which now instead gets sent to R5 and than to R4 however the ip that gets recorded is of the link between R5 and R4

    This actually than means that traceroute will not depict an accurate picture. Can you please confirm if this is normal and hence to get a normal accurate traceroute what needs to be done when troubleshooting.

    ReplyDelete
  5. Yes, this is normal. Things get even weirder as consecutive probes with the same TTL take different paths.

    Solution: don't do traceroute from the router but from a workstation attached to it.

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.