Paul Unbehagen made an interesting claim when presenting Avaya network built for Sochi Olympics during a recent Tech Field Day event: “we didn’t need MPLS or BGP to implement L2- and L3VPN. It was all done with SPB and IS-IS.”

Where’s the Magic?

IS-IS is a routing protocol that (like MP-BGP) supports multiple address families. We’ve used it to route IPv4 and IPv6 for over a decade, recently it got extended to support layer-2 routing with SPB and TRILL, and now Avaya is using it to transport L3VPN information.

The architecture of Avaya’s solution is really quite simple:

• They use SPB with MAC-in-MAC encapsulation to build large layer-2 fabrics;
• MAC-in-MAC encapsulation contains an I-SID field, which is sort-of equivalent to VLAN in 802.1q – you can use it to indicate L2VPN the encapsulated packet belongs to. Avaya uses that field to build L2VPNs on top of SPB fabric;
• IS-IS is used within SPB fabric to build L2 topology database, but there’s no reason not to use that same IS-IS to build IP routing topology database. Avaya used that approach for several years to add (global) IP routing functionality to SPB fabric;
• Recently Avaya added L3VPN functionality – new IS-IS TLVs to exchange VRF IP reachability information, and L3 forwarding based on I-SID field.

After SPB nodes exchange L2 and L3VPN reachability information (using regular IS-IS flooding) the packet forwarding follows (approximately) these steps:

• Incoming packet is received by ingress fabric switch;
• Destination MAC address of the incoming packet matches MAC address of switch’s IP interface – ingress fabric switch thus performs L3 lookup;
• Incoming packet is encapsulated in MAC-in-MAC envelope (with I-SID indicating L3VPN) and sent to the fabric transport MAC address of the egress fabric switch;
• Encapsulated packet is forwarded across the L2 SPB fabric to the egress switch based on the outer (fabric transport) MAC address
• Egress switch receives the encapsulated packet, selects local VRF based on I-SID value, performs L3 forwarding in selected VRF, and forwards the packet toward its final destination.

For more information, read Avaya’s Shortest Path Bridging configuration guide.

Will It Scale?

Avaya’s L3VPN solution is architecturally (almost) equivalent to MPLS/VPN and thus scales better than Cisco’s Easy Virtual Network (EVN), which is really just syntactic sugar on top of VRF-Lite.

The proof is left as an exercise for the reader, the solution can be found in opening chapters of MPLS and VPN Architectures book.

Even though there are architectural similarities, Avaya’s solution remains far away from true scalability of MPLS/VPN:

• SPB fabric runs a single-area IS-IS with all fabric members sharing the same topology database. The size of the fabric is thus limited by the weakest switch in the whole fabric;
• IS-IS implementations were traditionally better than OSPF implementations (that’s why many large ISPs prefer IS-IS over OSPF), but that doesn’t mean that you can grow an IS-IS area indefinitely. A few hundred switches (for a pretty low value of few) is probably the largest fabric you can build;
• The number of IP routes (carried as IS-IS TLVs) in enterprise networks is usually reasonably small, so I wouldn’t expect any scalability issues there. Furthermore, IS-IS considers the IP prefixes after the shortest tree has already been computed, so the computational complexity of IP route selection remains linear (O(n)).

Finally, BGP is a much richer protocol than IS-IS when it comes to routing policies. There are numerous arcane MPLS/VPN architectures that cannot be implemented with the simple L3VPN model Avaya is using. Admittedly, you wouldn’t find them in most enterprise networks.

Avaya’s SPB-based L3VPN implementation is pretty new, so tread carefully. For example, it seems route redistribution loops could cause major headaches (see configuration guide, page 173).

Summary

Avaya’s L3VPN solution seems a reasonable fit for enterprise networks that need L3 path separation (similar to the scenarios I described in Enterprise MPLS/VPN webinar), but I wouldn’t use it in large-scale service provider deployments.