You won’t find much about the QFabric forwarding architecture and resulting behavior in the documentation; white papers might give you more insight and I’m positive more detailed ones will start appearing on Juniper’s web site now that the product is shipping. In the meantime, let’s see how far we can get based on two simple assumptions: (A) The "one tier architecture" claim is true and (B) Juniper has some very smart engineers.
One lookup per packet. According to documentation available on Juniper’s web site and explanations given in the Juniper QFabric Packet Pushers Podcast, the whole QFabric acts as a single L2/L3 switch. Even more, only a single L2/L3 lookup is performed when a packet traverses the Qfabric. Obviously, the lookup has to be performed by the ingress QF/Node.
MAC address learning. In a traditional L2 network, every L2 switch would learn the paths toward end hosts by gleaning the source MAC addresses of transit packets. This approach can no longer work in one-lookup-per-packet environment; the ingress QF/Node can use dynamic MAC address learning, the egress ones cannot.
MAC address reachability information must thus be propagated by the control plane (similar to Cisco’s OTV). The control-plane mechanisms (most likely implemented in QF/Director) can distribute MAC addresses only to those QF/Nodes that are connected to the target VLAN; QFabric thus has the potential to scale much better than traditional L2 networks.
Layer-3 forwarding. Every QF/Node is a L2/L3 switch and has to be able to do full L3 forwarding between any two VLANs (even if the target VLAN is not configured on it) if you want to retain the one-lookup philosophy. The IP routing tables and ARP tables thus have to be shared between all QF/Nodes in QFabric.
To be more precise: QF/Node has to have IP routes and ARP tables for the routing instances in which its ports (based on their VLAN membership) participate.
The shared layer-3 forwarding architecture has an interesting consequence: every single IP packet is forwarded along the shortest physical path toward the destination IP host (or next-hop router outside of QFabric), regardless of whether the destination is in the same or in a different IP subnet. Good bye, L2/L3 separation and traffic trombones.
Combined with multipathing across and within QF/Interconnects, QFabric gives you optimum any-to-any L2 or L3 connectivity. Now I’m getting impressed.
Shared default router MAC address. Think about the L3 forwarding facts I mentioned above, the way off-subnet forwarding works with IP, and the mandatory support for host (actually VM) mobility.
An IP host usually reaches off-subnet destinations through the default gateway – that would be the ingress QF/Node in QFabric. The IP host reaches the default gateway via default gateway’s MAC address that it learns through ARP. A virtual machine (VM) cannot change the default gateway or its MAC address after a live migration event (that would require fixes to the guest OS TCP/IP stack); it’s still sending the off-subnet packets to the same MAC address (causing traffic trombones in traditional data center networks). To make this work with “one lookup in ingress QF/Node” policy, all QF/Nodes must share the same MAC address for the virtual IP address of the per-subnet (per-VLAN) default gateway.
Now think about the implications – every IP packet reaches the default gateway in one hop (good), the ingress switch always does the L3 lookup (better) ... and we don’t need VRRP any more (finally), as it doesn’t matter where the server sends the off-subnet IP packets – every single QF/Node is able to intercept them and perform L3 lookup.
Is this unique? Actually it is. Although other vendors could implement a similar solution (and it wouldn’t be impossible to make it work with traditional switching architectures), they usually prefer to play it safe and roll out incremental improvements of existing architectures (or dead ends like large-scale bridging).
I’ll talk about data center fabric architectures and networking requirements for cloud computing at the upcoming EuroNOG conference.
You’ll find in-depth discussions of various data center and network virtualization technologies in Data Center 3.0 for Networking Engineers webinar (buy recording or Data Center Trilogy). Fabric architectures from various networking vendors are described in the Data Center Fabric Architectures webinar (register). Both webinars are included in the yearly subscription.