Like anyone else, I was pretty impressed with the QFabric hardware architecture when Juniper announced it, but remained way more interested in the control-plane aspects of QFabric. After all, if you want multiple switches to behave like a single device, you could either use Borg-like architecture with a single control plane entity, or implement some very clever tricks.

Nobody has yet demonstrated a 100-switch network with a single control plane (although the OpenFlow aficionados would make you believe it’s just around the corner), so it must have been something else.

For example, you could run distributed OSPF processes that would appear as a single router from the topology database perspective while performing distributed computations and LSA flooding. Alternatively, you could start multiple independent OSPF processes, each one behaving like a regular router, and model the backplane (QF/Interconnect) as a LAN in the OSPF topology database.

Coming back to Earth, usually it makes sense to focus on typical use cases first, and the most typical data center use case is a large-scale server farm with a few upstream routers and external appliances (for example, firewalls). Juniper did exactly that and implemented two types of control plane functionality using server and network node groups.

Each QF/Node (QFX3500) switch can be an independent node, running its own Routing (and forwarding) Engine with a limited subset of control-plane protocols (LACP, LLDP/DCBX, ARP, IGMP snooping) required by a typical server deployment. The nodes communicate with the QF/Director-based fabric management and fabric control Routing Engines to get configuration and VLAN/L2/L3 forwarding information, but that’s it.

You can also group two QF/Nodes in a server node group to provide box redundancy through multi-chassis link aggregation (MLAG). The server group uses local Routing Engine (probably a typical Borg-like MLAG setup like Cisco’s VSS or HP’s IRF) and behaves exactly like the independent nodes – the same control-plane protocols (most importantly, LACP) work across two physical devices.

Last but definitely not least, you can combine up to eight QF/Nodes into a network node group, which uses redundant QF/Director-based Routing Engines in a setup very similar to XRE200/EX8200 combo. The network node group can run IP routing protocols (OSPF and BGP) and STP/RSTP/MSTP/VSTP.

The current QFabric implementation supports a single network node group; you can run routing protocols on up to 384 out of 6144 ports. It’s more than enough for the typical use case and most people probably don’t care, but there are few that still do their network designs the old-fashioned way and use loopbacks and RIP or OSPF on their UNIX hosts.

Summary

The architectural choices Juniper made are sound – you cannot run a single control plane spanning 100+ switches. Distributed control planes, single management/configuration plane, and internal control protocols (probably similar to those used in Virtual Chassis) make perfect sense.

However, after all the buzz and teasing blog posts, my inner geek is still not perfectly happy ... but my contacts within Juniper tell me what we’re seeing today is not the end of the story.

Disclosure: I got most of the information contained in this blog post from publicly available Junos for QFX Series, release 11.3 documentation. A phone call with Anjan Venkatramani, Raja Mukhopadhyay and Abner Germanow organized by Nadia Walker (all from Juniper) helped resolve the remaining mysteries like LLDP support.