OSPF Forwarding Address: Yet Another Kludge

One of my readers sent me an interesting NSSA question (more in a future blog post) that sent me chasing for the reasons behind the OSPF Forwarding Address (FA) field in type-5 and type-7 LSAs.

This is the typical scenario for OSPF FA that I was able to find on the Internet:

Two OSPF ASBRs are connected to the same external network. Only one runs the external routing protocol (BGP in my diagram) and redistributes external prefixes into OSPF. You’d want to send the traffic toward external destinations through both ASBRs, but you can’t do that as only one ASBR advertises the external routes.

Here are the relevant parts for E1 and E2 configuration. BGP (and redistribution into OSPF) is configured on E1 but not E2.

OSPF and BGP configuration on E1
interface Loopback0
 description Loopback
 ip address 192.168.0.1 255.255.255.255
 ip ospf 1 area 1
!
interface GigabitEthernet0/1
 description to C1
 ip address 10.0.128.1 255.255.255.252
 ip ospf 1 area 1
 ip ospf cost 1
!
interface GigabitEthernet0/2
 description to External_LAN
 ip address 10.0.0.1 255.255.128.0
!
router ospf 1
 redistribute bgp 65000 subnets
 passive-interface Loopback0
!
router bgp 65000
 bgp log-neighbor-changes
 redistribute ospf 1
 neighbor 10.0.0.2 remote-as 65001
OSPF configuration on E2
interface Loopback0
 description Loopback
 ip address 192.168.0.4 255.255.255.255
 ip ospf 1 area 1
!
interface GigabitEthernet0/1
 description to C1
 ip address 10.0.128.6 255.255.255.252
 ip ospf 1 area 1
 ip ospf cost 1
!
router ospf 1

And here’s the relevant part of the OSPF topology database. The prefix 192.168.0.2/32 that is advertised by X1 over EBGP is redistributed as type-5 LSA into OSPF by E1 (192.168.0.1)

Type-5 LSA in OSPF topology database
C1#show ip ospf database external

            OSPF Router with ID (192.168.0.3) (Process ID 1)

    Type-5 AS External Link States

  LS age: 301
  Options: (No TOS-capability, DC, Upward)
  LS Type: AS External Link
  Link State ID: 192.168.0.2 (External Network Number )
  Advertising Router: 192.168.0.1
  LS Seq Number: 80000001
  Checksum: 0xA154
  Length: 36
  Network Mask: /32
  Metric Type: 2 (Larger than any link state path)
  MTID: 0
  Metric: 1
  Forward Address: 0.0.0.0
  External Route Tag: 65001

Enter the YAK (Yet Another Kludge) twilight zone: what if the ASBR advertises the external next hop (forwarding address) in the type-5 LSA, and both ASBRs advertise the external prefix (hopefully as a stub network)? Recursive next-hop lookup would solve the problem, and C1 would get two equal-cost routes toward the external destinations.

To enable this functionality, E1 and E2 have to advertise the external subnet as an internal OSPF route, so we’ll add the external interface to the OSPF process on E1 and E2.

Enabling OSPF on the external interface on E1 and E2
interface GigabitEthernet0/2
 description to External_LAN
 ip ospf cost 1
It looks like on some versions of Cisco IOS, you cannot make the external interface passive (= stub network) – the routers would not set a forwarding address if the next-hop is on a passive interface; the external network must be a fully-functional OSPF interface. I was not able to find any relevant requirements in RFC 2328.

After the configuration change, the Type-5 LSA in the OSPF topology database gets the forwarding address value, and C1 gets two equal-cost paths to the external destination.

Type-5 LSA with Forwarding Address field set to external next hop
C1>show ip ospf database external

            OSPF Router with ID (192.168.0.3) (Process ID 1)

Type-5 AS External Link States

  LS age: 907
  Options: (No TOS-capability, DC, Upward)
  LS Type: AS External Link
  Link State ID: 192.168.0.2 (External Network Number )
  Advertising Router: 192.168.0.1
  LS Seq Number: 80000001
  Checksum: 0x2CBD
  Length: 36
  Network Mask: /32
Metric Type: 2 (Larger than any link state path)
MTID: 0
Metric: 1
Forward Address: 10.0.0.2
External Route Tag: 65001
Two equal-cost routes to external destination
C1>show ip route 192.168.0.2
Routing entry for 192.168.0.2/32
  Known via "ospf 1", distance 110, metric 1
  Tag 65001, type extern 2, forward metric 2
  Last update from 10.0.128.6 on GigabitEthernet0/2, 00:15:21 ago
  Routing Descriptor Blocks:
    10.0.128.6, from 192.168.0.1, 00:15:21 ago, via GigabitEthernet0/2
      Route metric is 1, traffic share count is 1
      Route tag 65001
  * 10.0.128.1, from 192.168.0.1, 00:15:21 ago, via GigabitEthernet0/1
      Route metric is 1, traffic share count is 1
      Route tag 65001

Problem solved? NO, OF COURSE NOT. You just made it worse:

  • Even though it looks like everything works as expected, E1 is a single point of failure – if it crashes, you lose route redistribution and connectivity to external destinations;
  • The already-too-complex link-state routing protocol got another hard-to-figure-out quirk. See the passive interface gotcha above, and check all the complications T5 FA caused in OSPF route selection process (RFC 2328);
  • To make this kludge work, you have to run OSPF on the external interface (at least with recent Cisco IOS releases). Not exactly the most secure design I’ve seen in my life (and please don’t even mention that you could use an ACL to filter incoming OSPF packets on the external interface before reading this blog post).

The proper design would be to run external routing protocol and route redistribution on both ASBRs (yeah, I know, the beauties of two-way redistribution), and tell everyone who complained about this deficiency of OSPF to get lost and fix his design.

Alas, that’s not how standards are made. No wonder academics are making fun of the complexities of distributed routing protocols, and we’re continuously involved in YAK shaving exercises.

12 comments:

  1. I was able to find only this info:
    * These conditions set the forwarding address field to a non-zero address:
    ** OSPF is enabled on the ASBR's next hop interface AND
    ** ASBR's next hop interface is non-passive under OSPF AND
    ** ASBR's next hop interface is not point-to-point AND
    ** ASBR's next hop interface is not point-to-multipoint AND
    ** ASBR's next hop interface address falls under the network range specified in the router ospf command.
    * Any other conditions besides these set the forwarding address to 0.0.0.0.

    http://www.cisco.com/c/en/us/support/docs/ip/open-shortest-path-first-ospf/13682-10.html#topic1
    Replies
    1. Yeah, I found that some thing... still wonder whether that's a Cisco thingy or RFC-mandated behavior.
    2. My (tested) solution:
      1) define area 1 as NSSA
      2) Configure on both E1 and E2 a common Loopback1 address (say 12.12.12.12/32)
      3) Include Loopback1 in OSPF 1 in both E1 and E2

      interface Loopback1
      ip addre 12.12.12.12 255.255.255.255
      ip ospf 1 area 1

      4) On E1 redistribute BGP in OSPF. Type-7 LSAs will have Forwarding Address 12.12.12.12

      That's all, folks !
  2. I spent on this topic some time and have found only a discussion ietf forum:
    https://www.ietf.org/mail-archive/web/ospf/current/msg08229.html
  3. One interesting use case is where the control plane is split from machines doing the actual data forwarding. If I am not mistaken, vmware NSX uses this on its logical routers, where the OSPF sessions are maintained by another VM.

    Other than that, I only came across this behavior when something went wrong on a network,.
    Replies
    1. That's correct. NSX does this as it must set the FA to the data plane device (the DLR) as the adjacency is acually on the controller. This is why when you set connected interfaces to redistribute it inserts them as type 5 or type 7. You cannot put them in the LSA database as anything but external which kind of sucks.
  4. Rather than make the interfaces passive you could maybe change the network type to something that doesn't dynamically discover neighbors. Prevents accidental adjacency and still lets you have ospf active on the interface. Maybe some consider that even kludgier, but I don't really consider this scenario that bad. I don't really mind any of the scenarios in this post, to be honest.

    As far as the academics, smart people of course, but sometimes their proposals are way of base. Seems like a lot of the stuff is either not fully functional, or end up looking like OSPF or BGP by another name. I took Nick Feamster's course once, and while interesting, most of the routing replacements he proposed were basically as bad or equal to kludgy static routing. Some parts of the course were cool but those were the parts that had nothing to do with the forwarding plane.
  5. RE:
    *The proper design would be to run external routing protocol and route redistribution on both ASBRs (yeah, I know, the beauties of two-way redistribution)*

    Seems like proper solution would include IBGP to avoid admin distance problems or stupid weight 32768 issues. Please correct if wrong:

    E1 ---(IBGP)--- E2
    E1 ---(EBGP)--- X1
    E2 ---(EBGP)--- X1
    Only redistribute one-way (BGP to OSPF)
    Use BGP network statements to advertise OSPF subnets to X1 or any other external AS
    Replies
    1. and I feel obliged to mention those admin distance / weight land mines are not present on other platforms
    2. Yes, that would be even better.
  6. Redistribute both E1 and E2 into OSPF and set the OSPF cost of the egress interfaces as equal or unequal based on active/active or active/passive forwarding. Do not matter if it is P2P or Multiaccess.
    Still OSPF sucks as LSP :) :P
  7. It might be worth mentioning that Forwarding Address is also required for optimal path selection in NSSA scenario with more than one ABR.
Add comment
Sidebar