Relationships between Layer-2 (VLAN) and Layer-3 (Subnet) Segments

Sometimes it takes me years to answer interesting questions, like the one I got in a tweet in 2021:

Do you have a good article describing the one-to-one relation of layer-2 and layer-3 networks? Why should every VLAN contain one single L3 segment?

There is no mandatory relationship between multi-access layer-2 networks and layer-3 segments, and secondary IP addresses (and subnets) were available in Cisco IOS in early 1990s. The rules-of-thumb1 claiming there should be a 1:1 relationship usually derive from the oft-forgotten underlying requirements. Let’s start with those.

The Basics

When you have a layer-3 host attached to a multi-access layer-2 network, that host has to:

  • Figure out which other layer-3 nodes are connected to the same segment so it can send the packets to them directly without going through an intermediate device.
  • Find out the layer-2 addresses of directly connected layer-3 nodes so it knows what to use as the destination layer-2 address in outgoing layer-2 frames.
  • Have a mechanism to find the next hop for destinations that are not directly connected.

Numerous solutions to these challenges have been developed in the past, including:

CLNS
  • Nodes have a single layer-3 address (NSAP).
  • There are no interface layer-3 addresses or subnets.
  • Directly connected nodes are discovered through end-system hellos (ESH) and intermediate-system hellos (ISH).
  • NSAP-to-layer-2 mappings are built from ESH/ISH information.
  • Routers advertise their presence with ISHs, hosts use one of the directly-attached routers to forward traffic to all nodes that are not directly connected (the host received no ESH from that node)
IPv4
  • Interfaces have IPv4 addresses and subnet masks.
  • Interface IPv4 address and subnet mask are used to figure out whether a destination IPv4 address is directly connected. Do a bitwise AND of subnet mask and source/destination IPv4 addresses. Destination is in the same subnet if the results are the same.
  • You can configure static routes pointing to an interface (without the next hop) to fake further directly-connected subnets. Some of us were stupid enough to do that with a default route, making the whole Internet directly connected. What could possibly go wrong?
  • Once the host decides a destination IPv4 address is directly connected, it uses ARP to find its MAC address.
  • Static routes (manually configured or derived from DHCP) are used for off-subnet forwarding. There is no standard mechanism to find the first-hop router.
IPv6
  • Interfaces have IPv6 addresses and prefix lengths. Prefix length is assumed to be /64; it can be changed via RA messages or static configuration.
  • Like in IPv4, IPv6 addresses and prefix lengths of statically configured interface addresses are used to figure out whether a destination IPv6 address is directly connected.
  • Prefixes included in IPv6 Router Advertisement messages are considered to be directly connected only if they have the on-link flag set.
  • IPv6 Neighbor Discovery is used to find the layer-2 address of directly-connected IPv6 nodes.
  • Router advertisement messages are used to find the first-hop router.

Notes

For even more information, watch the network addressing videos from How Networks Really Work webinar (available with free subscription).

Back to Best Practices

Now that you know what’s going on behind the scenes, it’s easy to figure out the reason for the 1:1 subnet-to-VLAN “best practice”:

  • If you have a single subnet stretched across multiple VLANs, hosts can’t reach each other because ARP/ND messages don’t cross VLAN boundaries. Obviously you can fix this one with proxy ARP/ND2 or LISP3.
  • If you have multiple subnets on a single VLAN, then (in the IPv4 world) the hosts have to send traffic to other hosts on the same VLAN through a router. Not a big deal if the router happens to be the core switch with linerate hardware L3 forwarding, but a major nuisance if the router is an overwhelmed underpowered device attached to the edge of the VLAN.

Having multiple IPv6 subnets per VLAN is easier. You can fix suboptimal packet forwarding with carefully-crafted RA messages, but as you probably won’t have more than a gazillion hosts per subnet4, it might be a better idea to keep things simple.


  1. Often called best current practices ↩︎

  2. … resulting in eternal appreciation of your coworkers who have to troubleshoot that monstrosity, and a permanent job security unless someone is smart enough to fire you before you manage to implement it. ↩︎

  3. There’s absolutely nothing you cannot fix with LISP (or NAT). SRv6 supposedly has similar magic properties. ↩︎

  4. 18446744073709551614 if you want to be more precise. ↩︎

4 comments:

  1. One of the biggest limitations I've seen in using multiple IPv4 subnets on a single interface was the fact that DHCP Relay can only be used for the primary subnet, as there is no way for the switch/router to distinguish in which of the configured subnets the host is requesting IP in.

    Replies
    1. The usual solution to that is called a "superscope" on the DHCP server (at least on MS DHCP servers). The server uses, e.g., the first available address from any of the subnets comprising the superscope.

      If specific hosts shall receive addresses from specific subnets, then this needs to be configured explicitly on the DHCP server (e.g., based on client ID).

    2. Daniel, DHCP server "shared network" and vendor specific tricks like "dhcp smart relay" can address the problem you mention.

  2. I think that a 1: 1 relationship between VLANs and subnets avoids the complexities of either using multiple subnets in a single VLAN or splitting a single subnet across several VLANs. Keeping things simple helps operating the network.
  3. Perhaps 1: 1 for each IP version? We can argue about "best", but every modern OS out there assigns one of each to every interface by default...
    Replies
    1. I agree that for Dual-Stack IPv4 and IPv6 there should usually be a 1: 1:1 relationship (1 VLAN, 1 IPv4 subnet, 1 GUA IPv6 subnet) unless there is a good reason for something else.

      IPv6 always adds link-local addressing, and there may be reasons for additional IPv6 on-link prefixes.

      IMHO needless complexity should be avoided.

  4. The sentence that „If you have multiple subnets on a single VLAN, then (in the IPv4 world) the hosts have to send traffic to other hosts on the same VLAN through a router.“ does not make sense to me. If I split these multiple subnets across multiple VLANs, the traffic still needs to be sent through the same router.

    Replies
    1. This is a great example regarding the problems created by ignoring this rule of thumb or best practice: what do people even mean when they make statements regarding the construct?

      In usual host configurations, the host knows about one local IP subnet and one default router. To send IP packets to a host in a different subnet, it uses the default gateway. The host usually does not know about VLANs or that it could send the data directly, because both subnets are configured on the same VLAN.

      But this could be changed via host configuration, such that hosts in different subnets, but the same VLAN, do communicate directly¹.

      Additionally, any host in the VLAN can see the other subnets via packet capture (e.g., via ARP frames). Thus the different IP subnets do not really separate the hosts in a VLAN, but by default the hosts do not communicate directly, i.e., they seem to be separated somehow.

      All this added complexity results in higher likelihood to have a wrong understanding and make mistakes.

      ¹ Example:

      • two hosts, A and B, connected to some VLAN with two IP subnets 192.0.2.0/24 and 198.51.100.0/24
      • Linux host A has IP address 192.0.2.42/24 using interface eth0
      • Linux host B has IP address 198.51.100.0/24 using interface eth0
      • add direct route on host A to the other subnet: ip route add 198.51.100.0/24 dev eth0
      • add direct route on host B to the other subnet: ip route add 192.0.2.0/24 dev eth0
      • now host A and B communicate directly without using a gateway
Add comment
Sidebar