The fundamental principle underlying OpenFlow and Software Defined Networking (as defined by Open Networking Foundation) is the decoupling of control- and data plane, with data (forwarding) plane running in a networking device (switch or router) and control plane being implemented in a central controller, which controls numerous dumb devices. Let’s start with the basics: what are data, control and management planes in a network device?

Every single network device (or a distributed system like QFabric) has to perform at least three distinct activities:

• Process the transit traffic (that’s why we buy them) in the data plane;
• Figure out what’s going on around it with the control plane protocols;
• Interact with its owner (or NMS) through the management plane.

Routers are used as a typical example in every text describing the three planes of operation, so let’s stick to this time-honored tradition:

• Interfaces, IP subnets and routing protocols are configured through management plane protocols, ranging from CLI to NETCONF and the latest buzzword – northbound RESTful API;
• Router runs control plane routing protocols (OSPF, EIGRP, BGP …) to discover adjacent devices and the overall network topology (or reachability information in case of distance/path vector protocols);
• Router inserts the results of the control-plane protocols into Routing Information Base (RIB) and Forwarding Information Base (FIB). Data plane software or ASICs uses FIB structures to forward the transit traffic.
• Management plane protocols like SNMP can be used to monitor the device operation, its performance, interface counters …

The management plane is pretty straightforward, so let’s focus on a few intricacies of the control and data planes.

We usually have routing protocols in mind when talking about Control plane protocols, but in reality the control plane protocols perform numerous other functions including:

• Interface state management (PPP, LACP);
• Connectivity management (BFD, CFM);
• Adjacent device discovery (hello mechanisms present in most routing protocols, ES-IS, ARP, IPv6 ND, uPNP SSDP);
• Topology or reachability information exchange (IP/IPv6 routing protocols, IS-IS in TRILL/SPB, STP);
• Service provisioning (RSVP for IntServ or MPLS/TE, uPNP SOAP calls);

Data plane should be focused on forwarding packets but is commonly burdened by other activities:

• NAT session creation and NAT table maintenance;
• Neighbor address gleaning (example: dynamic MAC address learning in bridging, IPv6 SAVI);
• Netflow Accounting (sFlow is cheap compared to Netflow);
• ACL logging;
• Error signaling (ICMP).

Data plane forwarding is hopefully performed in dedicated hardware or in high-speed code (within the interrupt handler on low-end Cisco IOS routers), while the overhead activities usually happen on the device CPU, often in userspace processes – -the switch from high-speed forwarding to user-mode processing is commonly called punting.

Typical Architectures

The management and control planes are typically implemented in a CPU, while the data plane could be implemented in numerous ways:

• Optimized code running on the same CPU;
• Code running on a dedicated CPU core (typical for high-speed packet switching on Linux servers);
• Code running on linecard CPUs (example: Cisco 7200);
• Dedicated processors (sometimes called NPUs)
• Switching hardware (ASICs);
• Switching hardware on numerous linecards.

Based on the capabilities of the switching hardware, a device might run some simple time-critical control-plane protocols (example: BFD) in hardware or on NPUs.

Processing of Inbound Control-Plane Packets

In most router implementations, the data plane receives and processes all inbound packets and selective forwards packets destined for the router (for example, SSH traffic or routing protocol updates) or packets that need special processing (for example, IP datagrams with IP options or IP datagrams that have exceeded their TTL) to the control plane.

The control plane might pass outbound packets to the data plane, or use its own forwarding mechanisms to determine the outgoing interface and the next-hop router (for example, when using the local policy routing).

Regardless of the implementation details, it’s obvious the device CPU represents a significant bottleneck (in some cases the switch to CPU-based forwarding causes several magnitudes lower performance) – the main reason one has to rate-limit ACL logging and protect the device CPU with Control Plane Protection features.