Whose Failure Domain Is It?

Draco made a valid comment to my Keep Your Failure Domain Small post:

What could a small ISP do to limit failure domains? Metro Ethernet and MPLS Virtual Private LAN service are all the rage, and offers customers the promise of being able to connect all their branch offices together, and use the same set of VLANs with free Layer 2 connectivity between their sites. It's either: extend the failure domains, or lose out in selling the service, b/c the customer will buy from another ISP.

Well, your customer’s failure domain doesn’t have to be yours.

Protect yourself

The first thing you should do as an ISP is to limit the amount of damage a customer could do to your edge (PE) routers:

  • Rate-limit the BUM frames entering your network (Cisco’s storm control is such a mechanism, although a pretty rudimentary one);
  • Configure BPDU guard (if appropriate), or root guard on customer-facing links;
  • Protect the PE router control plane.

Years ago, I met an ISP running RIPv1 with their customers (no, they were not offering MPLS/VPN services). Go figure ;)

Decouple customer networks from your core

If you implement a Metro Ethernet service with an end-to-end bridged network using 802.1Q or 802.1ad (QinQ) encapsulation, don’t act surprised when it collapses due to a L2 forwarding loop, or when you get uncontrollable amount of unicast flooding due to TCAM overflows – you’ve tightly coupled the fate of your core network to the (mis)behavior of your customers.

802.1ah (PBB) is slightly better – at the very least the core switches don’t see the customer MAC addresses any longer.

Pseudowires (EoMPLS or L2TPv3), VPLS or PBB-EVPN are the best solution – they totally decouple the layer-2 forwarding behavior of your services from stable layer-3 forwarding behavior of your IP+MPLS core. A problem within a customer’s network can no longer trash the ISP core (assuming the customer cannot generate enough traffic to overload the core links).

And now for a truly crazy idea

Hate MPLS and VPLS? Use VXLAN. Arista’s switches aren’t the best (or cheapest) ISP edge devices out there, but they do have wirespeed VXLAN VTEP, allowing you to build high-speed L2-over-L3 overlay networks.

1 comment:

  1. Ivan, I think you mentioned that VXLAN is not a DCI solution and now you are proposing that for metro-ethernet like scenario? Isnt it the same from "lack of loop prevention mechanism" perspective?

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.