Build the Next-Generation Data Center
6 week online course starting in spring 2017

Management, Control and Data Planes in Network Devices and Systems

Every single network device (or a distributed system like QFabric) has to perform at least three distinct activities:

  • Process the transit traffic (that’s why we buy them) in the data plane;
  • Figure out what’s going on around it with the control plane protocols;
  • Interact with its owner (or NMS) through the management plane.

Routers are used as a typical example in every text describing the three planes of operation, so let’s stick to this time-honored tradition:

  • Interfaces, IP subnets and routing protocols are configured through management plane protocols, ranging from CLI to NETCONF and the latest buzzword – northbound RESTful API;
  • Router runs control plane routing protocols (OSPF, EIGRP, BGP …) to discover adjacent devices and the overall network topology (or reachability information in case of distance/path vector protocols);
  • Router inserts the results of the control-plane protocols into Routing Information Base (RIB) and Forwarding Information Base (FIB). Data plane software or ASICs uses FIB structures to forward the transit traffic.
  • Management plane protocols like SNMP can be used to monitor the device operation, its performance, interface counters …

The slide is from one of my SDN/OpenFlow presentations

The management plane is pretty straightforward, so let’s focus on a few intricacies of the control and data planes.

We usually have routing protocols in mind when talking about Control plane protocols, but in reality the control plane protocols perform numerous other functions including:

  • Interface state management (PPP, LACP);
  • Connectivity management (BFD, CFM);
  • Adjacent device discovery (hello mechanisms present in most routing protocols, ES-IS, ARP, IPv6 ND, uPNP SSDP);
  • Topology or reachability information exchange (IP/IPv6 routing protocols, IS-IS in TRILL/SPB, STP);
  • Service provisioning (RSVP for IntServ or MPLS/TE, uPNP SOAP calls);

Data plane should be focused on forwarding packets but is commonly burdened by other activities:

Data plane forwarding is hopefully performed in dedicated hardware or in high-speed code (within the interrupt handler on low-end Cisco IOS routers), while the overhead activities usually happen on the device CPU (sometimes even in userspace processes – the switch from high-speed forwarding to user-mode processing is commonly called punting).

In reactive OpenFlow architectures a punting decision sends a packet all the way to the OpenFlow controller.

Regardless of the implementation details, it’s obvious the device CPU represents a significant bottleneck (in some cases the switch to CPU-based forwarding causes several magnitudes lower performance) – the main reason one has to rate-limit ACL logging and protect the device CPU with Control Plane Protection features.

More information

8 comments:

  1. From security perspective, when we look at a core router, if the control plane is well protected, will it really be possible to trigger some specific back-door (let's assume the extreme situation that router has government|vendor placed back-door) through data plane (transit traffic, things like some craft packets shutting the router down or starting to record some specific traffic and send them over to some receiver)? would like to know your take on that.

    ReplyDelete
    Replies
    1. If you believe there's a back door being buried deep in network device OS, it's always possible to trigger it with an innocuous-looking packet - something as simple as ping with a carefully crafted payload.

      Also, if I would be a device vendor, I wouldn't allow you to filter out my backdoor triggers with control plane protection, would I? ;)

      (* For everyone else: this is PURE UNFOUNDED SPECULATION *)

      Delete
  2. Shouldn't ICMP be more on the control-plane area?

    ReplyDelete
    Replies
    1. No, it's an inband response to a data-plane activity.

      Delete
  3. Ok, but it's still a control-plane generated response.
    Also, what about ICMPv6?

    ReplyDelete
    Replies
    1. I think you're confusing the ASIC/CPU split with data/control plane split.

      Even though punted traffic gets limited by what some vendors call CoPP, it doesn't transform punted traffic into control-plane protocols.

      Delete
  4. I'm actually talking about router responses to punted traffic, not the punted traffic itself.

    In my mind i visualize the idea of control-plane as a layer in the network device, where various processes run and these processes handle everything destined to the router itself. Maybe routing protocols have the control-plane functionality built-in, but still there are cases where for intermediate devices they are simple data-plane traffic.

    After all C=Control in ICMP.

    Quoting from yourself (http://wiki.nil.com/Control_and_Data_plane):

    In most router implementations, the data plane receives and processes all inbound packets and selective forwards packets destined for the router (for example, routing protocol updates) or packets that need special processing (for example, IP datagrams with IP options or IP datagrams that have exceeded their TTL) to the control plane.

    ReplyDelete
  5. Nice blog - I was asked to explain "SDN" to a DevOps meetup and ran them through this material as a reference so they could have context. The PPT for that is located here: http://www.slideshare.net/JeremySchulman/2014-0219-dev-ops-sdn-v1
    (disclaimer: I am by no means any expert on SDN)

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.