Unnumbered OSPF Interfaces in Quagga (and Cumulus)

Carlos Mendioroz sent me an interesting question about unnumbered interfaces in Cumulus Linux and some of the claims they make in their documentation.

TL&DR: Finally someone got it! Kudos for realizing how to use an ancient trick to make data center fabrics easier to deploy (and, BTW, the claims are exaggerated).

Since I wrote this blog post, Quagga went extinct (again), and was replaced by Free Range Routing (FRR) – the control plane daemons used in recent Cumulus Linux distributions

Why Am I Excited about Unnumbered Interfaces?

Every data center fabrics vendor claims how easy it is to configure their fabrics. Plug in the cables and the devices almost self-configure the fabric (or you might have to add a single configuration command like switchport mode fabric). Messing up the wiring? No problem, It All Just Works™.

These same vendors require you to configure IPv4 subnets on point-to-point links in a layer-3 leaf-and-spine fabric. Messing up the wiring? We wish you a happy troubleshooting!

Just in case you’re wondering why someone needs layer-3 leaf-and-spine fabrics: it’s not just overlay virtual networks, they could also be a large web property, doing finite element modelling, running a Hadoop cluster…

Can we get around the requirement of configuring IPv4 subnets on links that connect two routed interfaces on adjacent switches? Of course – we’ve been using unnumbered interfaces on point-to-point links for ages. It’s just that the routing protocol programmers haven’t realized the days of thick coax cable are gone; in this century most people use Ethernet on point-to-point links. There’s even a 6-year-old informational RFC describing this idea.

To be fair, some vendors did implement unnumbered Ethernet interfaces. For example, some Cisco IOS images have support for unnumbered VLAN and Ethernet interfaces, but cannot run routing protocols over them. Not exactly what I’ve been looking for. Does any other vendor fare better? Write a comment.

Did you notice I emphasized IPv4? IPv6 has unnumbered interfaces built in (they are called Link Local Addresses), and intra-AS routing protocols like OSPF have to use them (so it’s impossible to mangle your implementation in a way that would make numbered IPv6 point-to-point links mandatory).

And Now for the Claims

The Cumulus documentation claims:

In OSPFv2, configuring unnumbered interfaces reduces the links between routers into pure topological elements, and thus dramatically simplifies network configuration and reconfiguration. In addition, routing database contains only the real networks, hence memory footprint is reduced and SPF is faster.

Let’s walk through all of these claims:

Configuring unnumbered interfaces reduces the links between routers into pure topological elements

Translated into engineering terms: the Type-1 (router) LSA no longer contains the stub networks for inter-router subnets. You can do something similar on Cisco IOS with OSPF prefix suppression.

2014-06-24 17:40Z - Converting Ethernet interfaces into point-to-point links would also speed up adjacency establishment (no need for DR election) and get rid of type-2 LSAs connecting the two routers (as Enrique pointed out in the comments), which might result in marginally faster SPF (see below). You can achieve the same results in other OSPF implementations by configuring point-to-point OSPF interfaces.

… and thus dramatically simplifies network configuration and reconfiguration.

Marketese for “we don’t check IP subnets in OSPF hello packets”.

In addition, routing database contains only the real networks,

I don’t know what they call the routing database. OSPF database contains exactly the same number of LSAs, the routing table does contain smaller number of routes (but see also prefix suppression).

… hence memory footprint is reduced and SPF is faster.

Memory footprint is reduced. SPF speedup is probably measured in per mils – after all, the router considers the stub networks attached to Type-1 router LSAs only in the second part of the SPF algorithm, which has linear complexity.

Summary

I love the way FRR handles point-to-point Ethernet links – it does make the task of building a layer-3-only leaf-and-spine fabric much easier and error-resistant, but do you really have to call it dramatic?

20 comments:

  1. About "unnumbered" Interfaces, Cisco NX-OS doesn't support unnumbered interfaces, BUT support /31 interfaces that is a good compromise ...

    https://tools.ietf.org/rfc/rfc3021.txt

    Htam
    Replies
    1. /31 doesn't give you anything (apart from reduction in wasted addresses). Can you plug a L3-only link (routed interface) into another port and expect it to work?
    2. The person asking the question clearly didn't read what you wrote.
  2. But, doesn't this save one Type-2 Network LSA per point-to-point link in the OSPF database? It's a sincere question, I would say that the connectivity between both routers is included in their respective Type-1 LSAs and the Type-2 pseudonode is not required, saving some memory and processing.
    Replies
    1. There's that. Will reword the post. Thank you!
  3. Should one worry about debugging and monitoring if ip-unnumbered is used ?
    Replies
    1. Monitoring? Depends on the tool you use and what it uses as unique interface ID.

      Debugging? Of course there will be drawbacks (as always with the unnumbered interfaces). Example: traceroute will not identify individual transit interfaces, just boxes.
    2. For providing debugability of mis-cabled switches, we have PTMd:
      http://events.linuxfoundation.org/sites/events/files/slides/ptm_0.pdf

      We haven't implemented it yet in Cumulus Linux, but there is an RFC that can help with the traceroute issue:
      http://tools.ietf.org/html/draft-atlas-icmp-unnumbered-00
      you might recognize one of the former Googlers...
    3. PTMd is cool ;) Thanks for sharing the link!
  4. Would it be possible to achieve the same end goal using IS-IS as the routing protocol is decoupled from the IP? Do unnumbered interfaces do additional things to do with ARP?
    Replies
    1. I read the linked RFC and it answered my questions :D
  5. And yes, it is possible to do this even with BGP.
  6. IIRC 12.2(33)SRA train allows to do 'interface g1/1; ethernet point-to-point; ip unnumbered lo0' and runs happily OSPF over it (6500). 'ethernet point-to-point' was removed in later releases.
  7. IIRC 12.2(33)SRA allowed to do 'int g1/1; ethernet point-to-point; ip unnumbered lo0' and ran OSPF happily over it (6500). ethernet point-to-point was removed in the later releases.
  8. Junos supports unnumbered interfaces, and has for quite some time. It even supports unnumbered interfaces in MPLS topologies!
  9. You write 'I love the way Quagga handles point-to-point Ethernet links'. But I was not able to use unnumbered interfaces with stock Quagga in described way; two reasons for that:

    - an interface with netmask /32 address is not recognized as OSPF interface, even being point-to-point.

    - OSPF is choking on a host with several identical IP addresses (as you say here http://blog.ipspace.net/2008/10/ospf-breaks-when-faced-with-overlapping.html that "OSPF gets totally confused when two interfaces have the same IP address.")

    How come? Does Cumulus ship their own Quagga version to handle these two issues?
  10. They are constantly adding features to Quagga and pushing them back to Quagga distribution. Here's the description of this feature from Quagga Dev mailing list: https://lists.quagga.net/pipermail/quagga-dev/2013-October/010910.html

    See also http://docs.cumulusnetworks.com/display/CL25/Open+Shortest+Path+First+-+OSPF+-+Protocol#OpenShortestPathFirst-OSPF-Protocol-ConfigurationTip:UnnumberedInterfaces for details.
  11. > IPv6 has unnumbered interfaces built in (they are called Link Local Addresses), and intra-AS routing protocols like OSPF have to use them
    Yes, except for 'virtual interfaces', called so in RFC5340: "The IPv6 interface address of a virtual link MUST be an IPv6 address having global scope, instead of the link-local addresses used by other interface types."

    What can be the reason for this exception? And does it mean that if I have OSPFv3 running over a point-to-point IPv6 VPN link, then I must configure and "waste" a two-host IPv6 network?
    Replies
    1. OSPF virtual link is not an interface, but a construct in the OSPF topology database.

      VPN link is (usually) an interface, and standard interface numbering rules apply.
    2. Thank you very much for this clarification!! The confusion about "virtual link" was an expensive mistake here...
Add comment
Sidebar