Category: ARP

After figuring out ARP details, describing how routers use ARP to resolve entries in the IP routing table, and considering what we already know about OSPF on unnumbered IPv4 interfaces, we’re finally ready to answer Daniel’s question:

A few days ago, I described how ARP behaves when the source- and destination IP addresses are not on the same subnet (TL&DR: it doesn’t care). Now, let’s see how routers use ARP to get the destination MAC address for various entries in the IP routing table. To keep things simple, we’ll use static routes to insert entries in the IP routing table.

We’ll run our tests in a small virtual lab with two Linux hosts and an Arista vEOS switch. The link between H1 and RTR is a regular subnet. H2 has an IP address on the Ethernet interface, but RTR uses an unnumbered interface.

When figuring out how unnumbered IPv4 interfaces work, Daniel Dib asked an interesting question: How does ARP work when the source and destination IPv4 address are not in the same segment (as is usually the case when using unnumbered interfaces)?

TL&DR: ARP doesn’t care about subnets. If the TCP/IP stack needs to find a MAC address of a node it thinks is adjacent, ARP does its best, no matter what.

A reader sent me the following intriguing question:

I’m trying to understand the ARP behavior with SVI interface configured with anycast gateways of leaf switches, and with distributed anycast gateways configured across the leaf nodes in VXLAN scenario.

Without going into too many details, the core dilemma is: will the ARP request get flooded, and will we get multiple ARP replies. As always, the correct answer is “it depends” 🤷‍♂️

Decades ago there was a trick question on the CCIE exam exploring the intricate relationships between MAC and ARP table. I always understood the explanation for about 10 minutes and then I was back to I knew why that’s true, but now I lost it.

Fast forward 20 years, and we’re still seeing the same challenges, this time in EVPN networks using in-subnet proxy ARP. For more details, read the excellent ARP problems in EVPN article by Dmytro Shypovalov (I understood the problem after reading the article, and now it’s all a blur 🤷‍♂️).

Reading my Directed ARP and ICMP Redirects blog post you might have wondered “how did Directed ARP ever get into ***redacted***?”

I searched for “directed ARP cisco” and found this gem, which really talks about unicast ARP behavior, an ancient mechanism documented in RFC 1122 (it’s not my Google-Fu, I got the reference to RFC 1122 in this blog post).

One of my readers sent me this question:

When I did my ***redacted*** I encountered a question about Directed ARP. The RFC (https://tools.ietf.org/html/rfc1433) is in the "experimental" stage, and I found it really weird from ***** to include such a hidden gem in the ***redacted***.

Directed ARP is clearly one of those weird things that people were trying out in the early days of networking when packet forwarding and bandwidth were still expensive (read the RFC for more details), but I kept wondering “what exactly is going on when a host receives an ICMP redirect?” Time for a hands-on test.

John Jackson wrote an interesting comment on my Rearchitecting L3-Only Networks blog post:

What the host has configured for its default gateway doesn't really matter, correct? Because the default gateway in traditional L2 access networks really isn't about the gateway's IP address, but the gateway's MAC address. The destination IP address in the packet header is always the end destination IP address, never the default gateway.

He totally got the idea, however there are a few minor details to consider.

Enterasys implemented optimal layer-3 forwarding with an interesting trick: they support VRRP like any other switch vendor, but allow you to make all members of a VRRP group active forwarders regardless of their status.

Apart from a slightly more synchronized behavior, their implementation doesn’t differ much from Arista’s Virtual ARP, and thus shares the same design and deployment caveats.

For more information, watch the Fabric Routing video from the Enterasys Robust Data Center Interconnect Solutions webinar.

In the Optimal L3 Forwarding with VARP and Active/Active VRRP blog post I made a remark along the lines of “Things might get nasty [in Arista EOS Virtual ARP world] if you have configuration mismatches”, resulting in a lengthy and amazingly insightful email exchange with Lincoln Dale during which we ventured deeper and deeper down the Virtual ARP (VARP) rabbit hole. Here’s what I learned during out trip:

I’ve blogged about the need for optimal L3 forwarding across the whole data center in 2012 when I introduced it as one of the interesting requirements in Data Center Fabrics webinar. Years later, the concept became one of the cornerstones of modern EVPN fabrics, but there are still only a few companies that can deliver this functionality in a more traditional environment.

Keith sent me a set of Mobile ARP questions, starting with “What’s your view on using Mobile ARP in a large enterprise?”

Short summary: Mobile ARP is an ancient technology that was designed to solve a problem that disappeared with the deployment of DHCP. Now, let’s look at the bigger picture.

Judging by your comments, some of you have already faced a stupidity similar to the one I’ve described on Friday. The symptoms are well described in the comments: the CPU utilization of the ARP process increases, packet forwarding becomes sluggish and the router runs out of memory, potentially resulting in a router crash. Now let’s analyze what’s going on.

Years ago I was faced with an interesting challenge: an Internet customer was connected to our PE router with an Ethernet link and I did not want to include the PE router’s IP address in the default route on the CE router.

After pondering the problem for a while, I got a “brilliant” idea: if I would use an interface default route, proxy-ARP would solve all my problems. This is the configuration I’ve deployed on the CE-router: