Every single network device (or a distributed system like QFabric) has to perform at least three distinct activities:
- Process the transit traffic (that’s why we buy them) in the data plane;
- Figure out what’s going on around it with the control plane protocols;
- Interact with its owner (or NMS) through the management plane.
Routers are used as a typical example in every text describing the three planes of operation, so let’s stick to this time-honored tradition:
- Interfaces, IP subnets and routing protocols are configured through management plane protocols, ranging from CLI to NETCONF and the latest buzzword – northbound RESTful API;
- Router runs control plane routing protocols (OSPF, EIGRP, BGP …) to discover adjacent devices and the overall network topology (or reachability information in case of distance/path vector protocols);
- Router inserts the results of the control-plane protocols into Routing Information Base (RIB) and Forwarding Information Base (FIB). Data plane software or ASICs uses FIB structures to forward the transit traffic.
- Management plane protocols like SNMP can be used to monitor the device operation, its performance, interface counters …
The slide is from one of my SDN/OpenFlow presentations
The management plane is pretty straightforward, so let’s focus on a few intricacies of the control and data planes.
We usually have routing protocols in mind when talking about Control plane protocols, but in reality the control plane protocols perform numerous other functions including:
- Interface state management (PPP, LACP);
- Connectivity management (BFD, CFM);
- Adjacent device discovery (hello mechanisms present in most routing protocols, ES-IS, ARP, IPv6 ND, uPNP SSDP);
- Topology or reachability information exchange (IP/IPv6 routing protocols, IS-IS in TRILL/SPB, STP);
- Service provisioning (RSVP for IntServ or MPLS/TE, uPNP SOAP calls);
Data plane should be focused on forwarding packets but is commonly burdened by other activities:
- NAT session creation and NAT table maintenance;
- Neighbor address gleaning (example: dynamic MAC address learning in bridging, IPv6 SAVI);
- Netflow Accounting (sFlow is cheap compared to Netflow);
- ACL logging;
- Error signaling (ICMP).
Data plane forwarding is hopefully performed in dedicated hardware or in high-speed code (within the interrupt handler on low-end Cisco IOS routers), while the overhead activities usually happen on the device CPU (sometimes even in userspace processes – the switch from high-speed forwarding to user-mode processing is commonly called punting).
In reactive OpenFlow architectures a punting decision sends a packet all the way to the OpenFlow controller.
Regardless of the implementation details, it’s obvious the device CPU represents a significant bottleneck (in some cases the switch to CPU-based forwarding causes several magnitudes lower performance) – the main reason one has to rate-limit ACL logging and protect the device CPU with Control Plane Protection features.