Unnumbered Ethernet Interfaces
Imagine an Internet Service Provider offering Ethernet-based Internet access (aka everyone using fiber access, excluding people believing in Russian dolls). If they know how to spell security, they might be nervous about connecting numerous customers to the same multi-access network, but it seems they have only two ways to solve this challenge:
- Use private VLANs with proxy ARP on the head-end router, forcing the customer-to-customer traffic to pass through layer-3 forwarding on the head-end router.
- Use a separate routed interface with each customer, wasting three-quarters of their available IPv4 address space.
Is there a third option? Can’t we pretend Ethernet works in almost the same way as dialup and use unnumbered IPv4 interfaces?
Considering what we already know about unnumbered interfaces, it shouldn’t be too hard to get it done, considering that Ethernet forces you to supply next-hop layer-2 information even on point-to-point links.
Here’s what we have to do:
- We’d need a static host route on our access router for each client, pointing toward an unnumbered interface. The static routes could be preconfigured or created on the fly based on DHCP address allocation.
- We’d need ARP entries for client IP addresses connected to unnumbered interfaces to get the destination MAC address for the outbound IP packets.
It turns out that we had all the necessary bits and pieces for decades but couldn’t use them because network operating systems didn’t allow us to configure unnumbered multi-access interfaces.
Using Static Routes
We’ll start with the simplest scenario: using static routes. We’ll use a simple lab with an access router and two clients, each one of them connected to the access router with an Ethernet point-to-point link:
First, we need to configure a few unnumbered interfaces on the access router.
interface Loopback0
ip address 10.2.1.1 255.255.255.255
!
interface GigabitEthernet2
description rtr -> c1
ip unnumbered Loopback0
!
interface GigabitEthernet3
description rtr -> c2
ip unnumbered Loopback0
Next, we configure static routes for the client IP addresses pointing to those interfaces:
ip route 10.2.1.2 255.255.255.255 GigabitEthernet2
ip route 10.2.1.3 255.255.255.255 GigabitEthernet3
That’s it. Now you can log in to one of the clients and ping the other client.
How Does It Work?
Here’s the magic trick: whenever you configure a static route pointing to an interface (without a next-hop), the router assumes that any destination within the configured prefix is directly connected (even though all printouts claim it’s still a static route).
rtr#show ip route 10.2.0.0 longer-prefixes | begin Gateway
Gateway of last resort is 192.168.121.1 to network 0.0.0.0
10.0.0.0/32 is subnetted, 3 subnets
C 10.2.1.1 is directly connected, Loopback0
S 10.2.1.2 is directly connected, GigabitEthernet2
S 10.2.1.3 is directly connected, GigabitEthernet3
rtr#show ip route 10.2.1.2
Routing entry for 10.2.1.2/32
Known via "static", distance 1, metric 0 (connected)
Routing Descriptor Blocks:
* directly connected, via GigabitEthernet2
Route metric is 0, traffic share count is 1
How do you reach a directly connected IP host? You ARP for its IP address and use the MAC address you get back to send the packet to it… and it doesn’t matter if the destination IP address belongs to the same subnet or not. ARP will kick in as soon as a router forwards an IP packet using a route without a next-hop.
Here’s the (expected) ARP cache on our access router. The loopback IP address has an ARP entry on all unnumbered interfaces; the ARP entries for adjacent IP addresses (10.2.1.2 and 10.2.1.34) point to different interfaces.
rtr#show arp 10.2.0.0 255.255.0.0
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.2.1.1 - 5254.0049.6b64 ARPA GigabitEthernet2
Internet 10.2.1.1 - 5254.003a.5bba ARPA GigabitEthernet3
Internet 10.2.1.2 1 5254.00e3.3048 ARPA GigabitEthernet2
Internet 10.2.1.3 1 5254.00cb.bf02 ARPA GigabitEthernet3
Mission accomplished? Not exactly. While it’s possible to automate static route configuration, we still need to tell the clients what IP addresses to use. Time to add DHCP to the picture – the topic of the next blog post in this series.
Want to Know More?
- The configurations I used to generate the printouts are on GitHub.
- I discussed numerous aspects of network addressing in How Networks Really Work webinar.
How would it behave if you would use the IOS command "medium p2p" on the ethernet interfaces? How would the ARP table look like? What MAC addresses would be used to send packets?
Dear Anonymous,
The "medium p2p" interface configuration command is used on NX-OS. I didn't have to use it on Cisco IOS XE... as you would have discovered by looking at the device configurations (there's a reason I added a link to them at the end of the blog post).
As for "how would the printouts look like" - I don't have them handy, and I don't have time to collect them, but here's what you can do: set up a lab, and do the tests yourself. Nexus 9300v is downloadable as a Vagrant box, and you can use my topology file with netsim-tools, changing the default device type from csr to nxos.
Here's the documentation to get you started https://netsim-tools.readthedocs.io/en/latest/
Once you get those printouts, do get in touch and I'll add them to the blog post.
Thank you! Ivan
Cisco , Juniper etc have support for unnumbered ethernet style services for a long time. It was just wrapped up in the Broadband Subscriber services functions where they could charge extra for it and dynamic in nature.
It was long enough ago I was doing it on Cisco 7200s.
When I was testing it on a Cisco 7200 long long time ago, it only worked on VLAN subinterfaces (not on the main interface), and not with routing protocols... but yeah, that was exactly what we needed for Broadband Subscriber services.
Hi,
Really good to see your article on this. We have been using this in production for about 2 years now on the Cisco 4500x platform in vss.
The upside to all this is that migrating clients from a shared vlan to dedicated vlans and nothing from the client end routing or subletting had to change.
Of course we only use static routing here. The clients can not accidentally start arping for another client ip or take over the default gateway arp accidentally or intentionally and affect other clients.
The other plus is that we can apply per vlan policers in and out per client by vlan and not by IP.
We started to use this setup in our out-of-band network where production devices have their fxp0, em0 and what not interfaces connected (it's all Juniper based). Initially, OoB network was just a flat switched network, but then one day due to human error we had forwarding loop there and ... let's just say control plane of almost all devices stopped running the actual network as it was too busy to process lovely flood of ARP packets. It wasn't pretty. But back to unnumbered interfaces. We use EX3400 with QFX5100 for some aggregation in this out-of-band network. As it was mentioned, the best part is that from connected device's perspective, nothing has changed. We've enabled also proxy-arp on these unnumbered interfaces. For downstream routing it's just static routes pointing to each downstream interface through qualified-next-hop and for upstream routing (each access EX3400 is dual uplinked for redundancy) we use venerable RIP protocol. Uplink interfaces are actually configured as aggregated ethernet interfaces with fast LACP and just single member. That is in order to make not so obvious link failures fast for RIP. Of course, one can be a license and use something else than RIP and maybe even VRFs if the same network serves multiple tenants, but for us this is good enough. It's somewhat less elegant when we are connecting dual control plane devices, where each routing engine has its own IP address, but they also share third one. In this case, that third IP address must be able to move between two physical ports. And IRB interface can't be configured as unnumbered interface. Until we find better solution, we create local vlan dedicated for a given two switched downstream ports and loop it over two front ports (one is configured as vlan trunk and another as L3 interface with sub-interfaces). Three static routes then point to the L3 sub-interface. Also to make it less painful to configure, we have commit script which transforms macro applied at interface level to all necessary statements in various places (the actual interface configuration, static route(s) and for dual RE case also vlan, new sub-interface, etc.).
Arista Networks EOS 4.23.0F introduced support for IPv4 unnumbered interfaces. EOS 4.26.0.1F is the latest version as of this comment.
https://eos.arista.com/eos-4-23-0f/ipv4-unnumbered-interfaces/
Hello Ivan,
I tried a similar configuration in order to use three devices in a /30 segment. .1 and .2 devices where already running, my goal was to use the .0 (i am completly aware that using the .3 address was not a good idea). I used your configuration :
whereas the two other devices on the subnet have a regular configuration:
I was able to arp the neighbours, but unfortunately not to ping them, since the neighbours do not answer even if they correctly learn the arp entry :
I think the reason is to find inside the RFC 1812: > a router [...] (2) SHOULD silently discard on receipt (i.e., do not even deliver to applications in the router) any packet addressed to 0.0.0.0 or { <Network-prefix>, 0 }. If these packets are not silently discarded, they MUST be treated as IP broadcasts (see Section [5.3.5]). There MAY be a configuration option to allow receipt of these packets. This option SHOULD default to discarding them.
which matchs somehow the cef status on the neighbour :
Eventually, this can be bypassed by configuring the neighbours with the unnumbered trick :
Lab done, but not sure i want to deploy it :)
Note that on IOS & IOS-XE you can't use static routes pointing to interface without next-hop inside a VRF :
ip route vrf TEST 10.2.1.2 255.255.255.255 GigabitEthernet2
will raise an error : % For VPN or topology routes, must specify a next hop IP address if not a point-to-point interface
You can overcome this, either by configuring the interfaces with "medium p2p" (not available on classic IOS) or by using your route destination as next-hop :
ip route vrf TEST 10.2.1.2 255.255.255.255 GigabitEthernet2 10.2.1.2
which is accepted and seems to work perfectly... but it's weird and recursive :)
Thanks, bjonn, life saver! :D