Follow-up: Interface default route

Judging by your comments, some of you have already faced a stupidity similar to the one I’ve described on Friday (BTW, I’ve remembered this particular debacle when receiving a Pingsta case invitation with very similar symptoms). The symptoms are well described in the comments: the CPU utilization of the ARP process increases, packet forwarding becomes sluggish and the router runs out of memory, potentially resulting in a router crash. Now let’s analyze what’s going on.

What’s wrong: A static default route pointing to an interface indicates that the rest of the Internet is reachable through that interface as a single flat network. Every destination address is reachable directly through that interface.

Why does the problem occur on an Ethernet interface and not on a point-to-point link: The router does not need to know the layer-2 address of its neighbor on a point-to-point link; it has a single neighbor and the layer-2 encapsulation is static. On a multi-access interface the router has to match the destination IP address with a layer-2 address of the next-hop router or host. A static route pointing to an interface is an equivalent of telling the router that all the addresses covered by the static route’s IP prefix are directly connected.

How does the router react: Since every IP address on the Internet is supposedly directly connected, the router needs the layer-2 MAC address of every destination host if it wants to forward the IP traffic to it. For every new destination IP address the router sends an ARP request and drops all IP packets until the ARP request is resolved. This also explains the high CPU utilization caused by the ARP process.

Why is the packet forwarding so sluggish: If you’re not using CEF switching, the router falls back to process switching for every next-hop address that is not yet available in the ARP cache. The first few packets toward any new destination address on the Internet are therefore process switched.

Why is the router running out of memory: The ARP cache is not supposed to grow indefinitely. Cisco IOS has no mechanism that would limit its size, it’s flushing the ARP cache only when its entries age out (and even that is never going to happen if you use CEF).

5 comments:

  1. I worked on a similar problem years ago. A campus had set every desktop's default gateway to its own IP address & relied on proxy-arp for Internet connectivity.

    The campus core was an AGS+. It kept up OK until the combination of increased desktop count and increased Internet traffic buried it.

    --Mike

    ReplyDelete
  2. Glöm inte o combina :-E

    ReplyDelete
  3. What does this configuration result in if this is an IPv6 environment? No ARP broadcasts, does it use the IPv6 neighbors? Would the same sluggish performance be seen?

    ReplyDelete
  4. what was the solution

    ReplyDelete
    Replies
    1. Use a static default route with a next hop.

      Delete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.