Why is TRILL not routing at layer-2

Peter John Hill made an interesting observation in a comment to my “The TRILLing brain split” post; he wrote “TRILL really is routing at layer 2.”

He’s partially right – TRILL uses a routing protocol (IS-IS) and the TRILL protocol used to forward Ethernet frames (TRILL data frames) definitely has all the attributes of a layer-3 protocol:

  • TRILL data frames have layer-3 addresses (RBridge nickname);
  • They have a hop count;
  • Layer-2 next-hop is always the MAC address of the next-hop RBridge;
  • As the TRILL data frames are propagated between RBridges, the outer MAC header changes.

However, once the TRILL infrastructure is set up and the best paths are computed, bridging forwarding paradigms are used to forward host-to-host data traffic, including building MAC address table by listening to data traffic and flooding of packets sent to unknown MAC destination. TRILL therefore retains most of the non-scalable properties of transparent bridging with these exceptions:

  • Convergence is faster and more predictable;
  • Data forwarding can use all the available links;
  • Core RBridges (those that have no non-TRILL links) do not need to know the end-station MAC addresses;
  • Edge RBridges need to know end-station MAC addresses only for the VLANs in which they participate (but that’s also true in existing well-designed bridged networks).

For an in-depth overview of TRILL, start with the Setting the stage for TRILL, rethinking data center switching article by Brad Hedlund and continue with RFC 5556 (TRILL: Problem and Applicability Statement); you'll find a big-picture perspective in my Data Center 3.0 for Networking Engineers webinar (buy a recording or yearly subscription).

11 comments:

  1. It's a bit like MPLS, where the edge RBridges would be equivalent to a PE.

    ReplyDelete
  2. Indeed, the more I read about it the more is sounds like VPLS. But with IS-IS and outer MAC switching, instead of LDP/OSPF-TE/etc and label switch paths.

    ReplyDelete
  3. Peter John Hill21 July, 2010 22:40

    Wow, thanks for the excellent feedback. You are absolutely right that there are layer 2 issues that still exist with TRILL. The "unknown mac address" problem can be pretty serious. Nothing like sending lots of packets to an ip address with no mac address to start consuming some sort of router resource, whether it is CPU or some rate-limit for the gleans.

    Great blog, btw!

    ReplyDelete
  4. Petr Lapukhov22 July, 2010 05:17

    well since TRILL is about tunneling, it does truly route tunnel traffic based on encapsulation headers, but transparently bridges native ethernet frames at tunnel egress to maintain backward compatibility with classic ethernet bridging. Alas, nothing new and exciting - it's all been around forever :(

    ReplyDelete
  5. but look how excited all these marketing/sales people (Brad Hedlund post is a nice example). Now they can sell this old idea one more time.

    ReplyDelete
  6. Ivan,
    In this article you say the outer MAC changes as it traverses RBridges. In Cisco's "proprietary" implementation of TRILL (called FabricPath), that is not true. With Cisco FabricPath, the outer MAC is that of the far end Edge bridge and does not change as it traverses RBridges in the core. From a "standards" perspective, in reading the spec for 802.1aq (SPBM, the newer variant that uses mac-in-mac) there is no language I can see that says the outer MAC should change when passing through RBridges (correct me if you see otherwise). This very different from L3 "Routing" behavior where the L2 destination address changes link by link.

    This further re-enforces your point that saying "TRILL really is routing at Layer 2", while a cute and semi-accurate sound-bite, is not completely true when you begin to peel back the onion.

    Thanks for the link, Ivan. I really appreciate that.

    Cheers,
    Brad
    :)

    ReplyDelete
  7. Ivan Pepelnjak27 July, 2010 08:43

    Highly interesting, thanks for the info. Now I have to figure out how they've solved the ECMP problem without hop-by-hop L2 addresses (it's trivial if you change L2 addresses with every hop).

    ReplyDelete
  8. Ivan Pepelnjak27 July, 2010 19:31

    Brad,

    another question (if you can share the answer): is FabricPath similar to SPBM or SPBV?

    Thanks,
    Ivan

    ReplyDelete
  9. Ivan,
    Cisco FabricPath is similar to 802.1aq SPBM, with a couple of additional enhancements:
    1) a more optimized Conversation based MAC learning
    2) identifying multiple multicast topologies, not just one, and load balancing multicast traffic across each topology on a per group basis.

    Cheers,
    Brad

    ReplyDelete
  10. Ivan,
    The Rbridges exchange their topology (links) and identity (MAC address) through IS-IS. Therefore if a "core" RBridge has equal cost paths to an "edge" RBridge, it will be known ahead of time which of its links are included in those paths. When a "core" RBridge receives a frame with a MAC of an edge bridge (the outer MAC) it will apply the ECMP logic and send traffic out each link on a per flow basis.

    Cheers,
    Brad

    ReplyDelete
  11. Ivan,
    The Rbridges exchange their topology (links) and identity (MAC address) through IS-IS. Therefore if a "core" RBridge has equal cost paths to an "edge" RBridge, it will be known ahead of time which of its links are included in those paths. When a "core" RBridge receives a frame with a MAC of an edge bridge (the outer MAC) it will apply the ECMP logic and send traffic out each link on a per flow basis.

    Cheers,
    Brad

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.