Category: ARP

In the Optimal L3 Forwarding with VARP and Active/Active VRRP blog post I made a remark along the lines of “Things might get nasty [in Arista EOS Virtual ARP world] if you have configuration mismatches”, resulting in a lengthy and amazingly insightful email exchange with Lincoln Dale during which we ventured deeper and deeper down the Virtual ARP (VARP) rabbit hole. Here’s what I learned during out trip:

I’ve blogged about the need for optimal L3 forwarding across the whole data center in 2012 when I introduced it as one of the interesting requirements in Data Center Fabrics webinar. Years later, the concept became one of the cornerstones of modern EVPN fabrics, but there are still only a few companies that can deliver this functionality in a more traditional environment.

Keith sent me a set of Mobile ARP questions, starting with “What’s your view on using Mobile ARP in a large enterprise?”

Short summary: Mobile ARP is an ancient technology that was designed to solve a problem that disappeared with the deployment of DHCP. Now, let’s look at the bigger picture.

Judging by your comments, some of you have already faced a stupidity similar to the one I’ve described on Friday. The symptoms are well described in the comments: the CPU utilization of the ARP process increases, packet forwarding becomes sluggish and the router runs out of memory, potentially resulting in a router crash. Now let’s analyze what’s going on.

Years ago I was faced with an interesting challenge: an Internet customer was connected to our PE router with an Ethernet link and I did not want to include the PE router’s IP address in the default route on the CE router.

After pondering the problem for a while, I got a “brilliant” idea: if I would use an interface default route, proxy-ARP would solve all my problems. This is the configuration I’ve deployed on the CE-router:

The Wake-on-LAN discussion we had a while ago brought us nowhere; there's simply no way to generate UDP packets on the router. I thought I could use Application Performance Monitor's Tcl scripts to generate the packet, but it looks like APM has been removed from recent IOS releases (and it's not clear whether you can use APM without a peer router).

The discussion nonetheless had an interesting side effect. Robert Turnšek sent me an interesting trick: with static ARP you can generate layer-2 broadcasts with a layer-3 unicast packet.

Under some circumstances, you might want to tune the ARP timers on the router (for example, when using ARP as a keepalive mechanism to detect whether the host is up). Unfortunately, although you can set the per-interface arp timeout in seconds, the actual timer resolution is in minutes. For example, if you set the ARP timeout to 10 seconds, the router will age the ARP entries once per minute.

In a previous post I've been writing about the inability to clean the ARP cache due to cached CEF adjacencies. As it turns out, this behavior has another side effect: the router will automatically refresh all ARP entries (and CEF adjacencies) as they expire from the ARP cache. This might become a problem on high-end devices with a lot of directly connected hosts if you set the arp timeout to a low value.

Whenever a router running CEF switching has LAN interfaces (or any other multi-access interfaces), you'll find cached adjacencies for active directly attached IP neighbors in its CEF table. These adjacencies ensure the smooth traffic flow toward the LAN-attached next-hops (preventing the initial packet drop symptom once the next-hop becomes active).