A few years ago Cisco introduced an interesting concept to the data center networking: fabric extenders, devices acting like remote linecards of a central switch (Juniper’s “revolutionary” QFabric looks very similar from a distance; the only major difference seems to be local switching in the QF/Nodes). Cisco’s proprietary technology used in its FEX products became the basis for 802.1Qbh, an IEEE draft that is supposed to standardize the port extender architecture.

If you’re not familiar with the FEX products, read my “Port or Fabric Extenders?” article before continuing ... and disregard most of what it says about 802.1Qbh.

The core ideas behind 802.1Qbh are very simple:

• After power-up, the port extender finds its controlling bridge (connected to the upstream port)
• Port extender tells the controlling bridge how many ports it has;
• The controlling bridge creates a logical interface for each port extender port and associates a tag value with it;
• Port extender tags all packets received trough its ports with tags assigned by the controlling bridge;

802.1Qbh uses its own encapsulation format to allow transparent transport of 802.1Q, 802.1ad and 802.1ah frames between port extender and controlling bridge.

• All traffic between ports on the same port extender goes through the controlling bridge that can perform traffic inspection and QoS actions if needed.

One of the beauties of the 802.1Qbh architecture is also its ability to support a port extender hierarchy – an upstream port of a port extender can connect to an external port on another port extender (during the initialization process, the external port becomes a cascade port).

Unfortunately, 802.1Qbh seems to be stuck in the standardization process. Initially 802.1Qbh specified just the control protocol and relied on 802.1Qbg (802.1ad) encapsulation. Later on it became a full-blown draft with its own encapsulation mechanism, control protocol and forwarding behavior. The latest draft is totally mangled; it seems like the working group decided the port extender functionality deserves another document (802.1BR) and now some people say most of the functionality can be implemented with PBB-TE (802.1Qay) anyway.

Anyhow, when Cisco decided to use virtual NICs in UCS, they reused their fabric extender technology (or so they claim) and renamed it to VN-Tag. With the VN-Tag technology (implemented in Palo adapter), a single physical NIC can appear as multiple NICs to the operating system, with each NIC acting as an independent Ethernet adapter. A logical port in the 6100 switch is associated with each virtual NIC in the Palo adapter and VN-Tag is used to tag frames, allowing the Palo adapter and the switch to place them in the right input queues.

What’s the difference between VN-Tag/802.1Qbh and S-component of EVB(802.1Qbg)

802.1Qbh (in its current form; who knows how the standard will look like in the end) allows any encapsulation (including 802.1Q, 802.1ad and 802.1ah) to be used on top of a logical point-to-point link. S-component of EVB is able to transport only untagged or 802.1Q-tagged frames.

802.1Qbh (assuming it doesn’t change too much) is also better at handling multicast replication. Port extenders can perform remote replication (replication of frames within port extender to multiple ports), whereas EVB bridge might have to send multiple copies of the same packet to the EVB host (one over each S-channel).

However, 802.1Qbg will fare better in virtualized environments (once the hypervisor vendors eventually implement it in their vSwitches), as it allows the hypervisor to tell the EVB bridge what it needs (for example, which VLAN to create). 802.1Qbh has no such protocol; a port extender (including an UCS blade server) can tell its controlling bridge how many ports (virtual NICs) it has but not how they should be configured.

More information

The details of VMware networking are described in VMware Networking Deep Dive webinar. If you want to learn more about modern data center architectures, watch Data Center 3.0 for Networking Engineers webinar. Both webinars are also part of the yearly subscription package.