Implementing Control-Plane Protocols with OpenFlow

The true OpenFlow zealots would love you to believe that you can drop whatever you’ve been doing before and replace it with a clean-slate solution using dumbest (and cheapest) possible switches and OpenFlow controllers.

In real world, your shiny new network has to communicate with the outside world … or you could take the approach most controller vendors did, decide to pretend STP is irrelevant, and ask people to configure static LAGs because you’re also not supporting LACP.

Hybrid-mode OpenFlow with traditional control plane

If you’re implementing hybrid-mode OpenFlow, you’ll probably rely on the traditional software running in the switches to handle the boring details of control-plane protocols and use OpenFlow only to add new functionality (example: edge access lists).

Needless to say, this approach usually won’t result in better forwarding behavior. For example, it would be hard to implement layer-2 multipathing in hybrid OpenFlow network if the switches rely on STP to detect and break the loops.

OpenFlow-based control plane

In an OpenFlow-only network, the switches have no standalone control plane logic, and thus the OpenFlow controller (or a cluster of controllers) has to implement the control plane and control-plane protocols. This is the approach Google used in their OpenFlow deployment – the OpenFlow controllers run IS-IS and BGP with the outside world.

OpenFlow protocol provides two messages the controllers can use to implement any control-plane protocol they wish:

  • The Packet-out message is used by the OpenFlow controller to send packets through any port of any controlled switch.
  • The Packet-in message is used to send messages from the switches to the OpenFlow controller. You could configure the switches to send all unknown packets to the controller, or set up flow matching entries (based on controller’s MAC/IP address and/or TCP/UDP port numbers) to select only those packets the controller is truly interested in.

For example, you could write a very simple implementation of STP (similar to what Avaya is doing on their ERS-series switches when they run MLAG) where the OpenFlow controller would always pretend to be the root bridge and shut down any ports where inbound BPDUs would indicate someone else is the root bridge:

  • Get the list of ports with Read State message;
  • Send BPDUs through all the ports claiming the controller is the root bridge with very high priority;
  • Configure flow entries that match the multicast destination address used by STP and forward those packets to the controller;
  • Inspect incoming BPDUs, and shut down the port if the BPDU indicates someone else claims to be a root bridge.

Summary

OpenFlow protocol allows you to implement any control-plane protocol you wish in the OpenFlow controller; if a controller does not implement the protocols you need in your data center, it’s not due to lack of OpenFlow functionality, but due to other factors (fill in the blanks).

If the OpenFlow product you’re interested in uses hybrid-mode OpenFlow (where the control plane resides in the traditional switch software) or uses OpenFlow to program overlay networks (example: Nicira’s NVP), you don’t have to worry about its control-plane protocols.

If, however, someone tries to sell you software that’s supposed to control your physical switches, and does not support the usual set of protocols you need to integrate the OpenFlow-controlled switches with the rest of your network (example: STP, LACP, LLDP on L2 and some routing protocol on L3), think twice. If you use the OpenFlow-controlled part of the network in an isolated fabric or small-scale environment, you probably don’t care whether the new toy supports STP or OSPF; if you want to integrate it with the rest of your existing data center network, be very careful.

More information

To find more about OpenFlow, watch our OpenFlow Deep Dive webinar; you might also be interested in How Networks Really Work.

6 comments:

  1. Hi Ivan,
    nice blog post but isn't the OF "Normal" forwarding action supposed to be a solution for such problems? Can't you use a fancy OF-only island and on the edge-ports to the outside world let the standard control-plane handle any protocols needed via the mentioned "Normal" forwarding action? As we probably won't see any Google-style OF white-box switches for the standard customer soon due to the lack of network programming skills needed for an appropriate implementation this sounds like a perfect solution to me.
    Replies
    1. OF-normal is definitely a workaround, but the question then becomes whether the controller supports it or not. In some cases that I'm familiar with the answer is "no".
    2. I wouldn't necessarily say it is just a workaround but rather see it on par with an OpenFlow-based control plane.
      But unfortunately you are right and the support is the crucial factor here. What I do not get is why you see the problem on the controller side only? Maybe you can disclose which controllers you are referring to because FloodLight for example definitely supports it. I am not sure about ODL(Cisco contribution) because they don't mention "normal" but "software and hardware path". Maybe someone can point out the exact meaning of this. But you should also consider the switch hardware to be capable of OF-normal forwarding. As this action is unfortunately only optional even in the OF 1.3.2 spec you can't blame the vendors but up to now I have only seen Quanta switches(also the basis for Pica8)to be capable of OF-Normal.
      Let's see how this will work out in the future ;-)
  2. Mechanisms that we have used forever to control the sanity of our network will continue to have a place in SDN networks (OF or otherwise). What they control, how they control it, and which ones will be absorbed into controller software in some fashion is what will need to be determined. The way we control networks will change dramatically, many of the things we need to control will still be the same.

  3. Ivan,

    You have great content and I enjoy your perspective. For many complex topics, you have the ability to boil it down to something understandable for people (including myself) visiting your site.

    However, you mention:
    "For example, you could write a very simple implementation of STP (similar to what Avaya is doing on their ERS-series switches when they run MLAG)"

    Just want to be clear that Avaya's MLAG is not using STP. MLAG was a term coined in recent years to reflect "Multi Chassis Link Aggregation". Multi Chassis Link Aggregation was originally developed in 2001 by Nortel (now Avaya) called Split Multi-Link Trunking (SMLT). This was even before the ratification of the IEEE LACP standard for single switch/chassis Link Aggregation.

    While link aggregation between physically separate core switches is relatively new (within the past 5 years) for other vendors, Avaya's (Nortel heritage) SMLT technology provided interoperability with pre-LACP technologies such as Cisco's EtherChannel/GigEtherChannel as well as LACP when it became ratified. Given the age of support, it also meant enhancements in real-world environments as well as support across a range of products (not just high end chassis).

    It's important not to diminish the roots of this technological advancement as it was the first technology to remove Spanning Tree from the core of the network. This is something that various forms of Data Center Fabrics are proposing today to eliminate.

    In 2001, building a high availability network involved connecting edge switches over Ethernet to 2 x separate core switches that were also connected together over Ethernet. Although not originally intended as a redundancy protocol, rather a path selection tool to avoid loops, Spanning Tree did prevent loops and offered a means of automated restoration in the event of failure.

    While other issues existed with Spanning Tree Protocol, relating to network stability, two of the main drawbacks were time to convergence and blocking of redundant paths. Spanning Tree convergence times were ~45 seconds in a small network and increased as the network grew. Redundant links sat idle waiting to process traffic in the event of a failure.

    Understanding the importance of carrying real-time protocols (voice/video), Spanning Tree was clearly not the protocol of choice for sub-second restoration times. In addition, links going unused waiting for failure scenarios was a waste of available bandwidth. Taking an application centric approach, as opposed to a network centric approach, SMLT was created to adapt the network to the application needs (sub second fail-over, utilization of all links and compliant with standard and proprietary LAG methods used by equipment attached). The result was/is a technology deployed in some of the largest Data Center's around the globe and what other vendors patterned their implementations after.
    Replies
    1. Thanks for the comment!

      I'm well aware of all the benefits of SMLT (and MLAG) - do a search on MLAG and you'll find plenty of posts describing its benefits (and there's even more in DC 3.0 and DC Fabrics webinars).

      Unfortunately Nortel never got it right - you couldn't run STP over SMLT for a long time, and then you could, but only if you were a root bridge. Other vendors had STP-over-MLAG support years before Nortel/Avaya did.

      You might argue that you don't need STP with MLAG ... and that might be true IF AND ONLY IF there is zero chance of getting a loop somewhere in the network. I wouldn't bet my network on that.
Add comment
Sidebar