Reinventing Your Own STP Wheel...

One of my readers sent me a link to an interesting L2-over-IP "design". Someone tried to connect two data centers with redundant etherip links using home-brewed redundancy mechanism and (surprise, surprise) managed to bring both of them down. The obvious fix: patch the etherip device driver.

EtherIP is pre-VXLAN Ethernet-over-IP technology yet again proving RFC1925 Rule 11.

I don't know enough about OpenBSD to figure out whether (A) it doesn't have STP at all, (B) STP doesn't work over EtherIP, (C) host routing based on ARP entries would be too much of a hassle, (D) some people don't understand the networking fundamentals, (E) everything looks like a nail once you found a hammer, or (F) all of the above. Insightful comments would be highly appreciated.

8 comments:

  1. Remi 15 August 2019 22:03
    OpenBSD has support for STP. It's part of the bridge implementation. The manual for OpenBSDs bridge explains why STP does not work with etherip pseudo interfaces: they lack a hardware MAC address (http://man.openbsd.org/bridge).

    There is a continuation to that story here: https://marc.info/?l=openbsd-tech&m=156445794826074&w=2
    Replies
    1. Remi 16 August 2019 08:02
      Apparently this limitation of the etherip interface does not exist anymore: https://marc.info/?l=openbsd-cvs&m=156592053422209&w=2
    2. Ivan Pepelnjak 16 August 2019 10:01
      I always thought you don't need a MAC address per interface (just one per box) in a bridge.
    3. dlg 19 August 2019 00:56
      The impression I get from the spec is that each individual port has a MAC address. The bridge has one too, but may reuse the MAC address from one of the ports if it wants.
    4. Ivan Pepelnjak 19 August 2019 15:17
      The way I read section 8.13.2, there MUST be a MAC address per MAC Service (= bridge, see section 6.2) and there COULD be a MAC address per bridge port which COULD be used for outgoing frames UNLESS specified otherwise.

      However, I think we should move this discussion to email. Remi knows how to get in touch with me (and I'm guessing you know how to get in touch with Remi ;).
  2. dlg 16 August 2019 08:33
    (A) is not true, (B) is not true, (C) is true, (D) is quite likely, and (E) is definitely true.

    FWIW, one side of this link was going into FEXes on Cisco Nexus kit, and they don't appear to like doing spanning tree out the front of those ports. Also, the first attempt at this had pairs of boxes at each site driving a single etherip link between them, which made it hard for STP to do something useful.

    The current setup has been very robust. I'm just sorry to have upset you by showing an intermediate step while I was learning some hard lessons.
    Replies
    1. Ivan Pepelnjak 16 August 2019 10:00
      Thanks for the feedback... and yes, FEXes (and rbridges and LAN extenders and a gazillion similar kludges) should never have been invented.

      Also, I always appreciate people documenting the failures - that's the only way for everyone to learn and move forward without repeating the same mistakes.
    2. lcw 11 September 2019 18:09
      FEX's always block spanning-tree packets (BPDUs).

      Too many network designers think LAN extension is the only way to do IP mobility. That's a shame because IP mobility is delightfully easy and works for almost all network traffic. (the most common exception is multicast & broadcast traffic traffic like heartbeats and discovery protocols -- those still require full LAN extension)
Add comment
Home
Sidebar