Controller-Based Packet Forwarding in OpenFlow Networks

One of the attendees of the ProgrammableFlow webinar sent me an interesting observation:

Though there is separate control plane and separate data plane, it appears that there is crossover from one to the other. Consider the scenario when flow tables are not programmed and so the packets will be punted by the ingress switch to PFC. The PFC will then forward these packets to the egress switch so that the initial packets are not dropped. So in some sense: we are seeing packet traversing the boundaries of typical data-plane and control-plane and vice-versa.

He’s absolutely right, and if the above description reminds you of fast and process switching you’re spot on. There really is nothing new under the sun.

OpenFlow controllers use one of the following two approaches to switch programming (more details @ NetworkStatic):

  • Proactive flow table setup, where the controller downloads flow entries into the switches based on user configuration (ex: ports, VLANs, subnets, ACLs) and network topology;
  • Reactive flow table setup (or flow-driven forwarding), where the controller downloads flow entries into the switches based on the unknown traffic the OpenFlow switches forward to the controller.

Even though I write about flow tables, don’t confuse them with per-flow forwarding that Doug Gourlay loves almost as much as I do. A flow entry might match solely on destination MAC address, making flow tables equivalent to MAC address tables, or it might match the destination IP address with the longest IP prefix in the flow table, making the flow table equivalent to routing table or FIB.

Proactive (predictive?) flow setup works only when the controller knows the topology of the network and all the endpoint addresses (MAC addresses, IP addresses or IP subnets). If you’d have an OpenFlow controller emulating OSPF or BGP router, it would be easy to use proactive flow setup; after all, the IP routes never change based on the application traffic observed by the switches.

Intra-subnet L3 forwarding is already a different beast. One could declare ARP/ND to be an authoritative control-plane protocol (please don’t get me started on the shortcomings of ARP and whether ES-IS would be a better solution) in which case you could use proactive flow setup to create host routes toward IP hosts (using an approach similar to Mobile ARP – what did I just say about nothing being really new?).

However, most vendors’ marketing departments (with a few notable exceptions) think their gear needs to support every bridging-abusing stupidity ever invented, from load balancing schemes that work best with hubs to floating IP or MAC addresses used to implement high-availability solutions. End result: the network has to support dynamic MAC learning, which makes OpenFlow-based networks reactive – nobody can predict when and where a new MAC address will appear (and it’s not guaranteed that the first packet sent from the new MAC address will be an ARP packet), so the switches have to send user traffic with unknown source or destination MAC addresses to the controller, and we’re back to packet punting.

Bridges (lovingly called layer-2 switches) don’t punt packets with unknown MAC addresses to the CPU; dynamic MAC address learning and unknown unicast flooding is implemented in hardware ... but that’s not how OpenFlow is supposed to work.

Within a single device the software punts packet from hardware (or interrupt) switching to CPU/process switching, in a controller-based network the switches punt packet to the controller. Plus ça change, plus c'est la même chose.


  1. Ian,

    Great post as always. Question, is the "proactive" and "reactive" options open to vendor implementations? We have all seen the negative affects punting traffic has, so the OF controller would have to have some sort of control-plane policing (CoPP) implemented in the case of reactive-mode, or one, or a network anomoly could easily DDoS the controller.
    1. Of course it depends - on the functionality the controller vendor wants to implement, on the controller implementation (some open-source controllers are notoriously simplistic) and the capabilities of the physical switch (vanilla OpenFlow lacks too many features).

      CoPP has to be implemented in linecards (oops, OpenFlow switches). NEC has something similar to CoPP (unicast flooding limit), but that functionality is not part of OpenFlow, so they had to implement a proprietary extension that only works on their switches.
  2. I think DHT-based flow routing makes more sense than dest-ip (gateway) based routing on an openflow network.
Add comment