SR-MPLS or SRv6 for Greenfield Networks
Here’s an interesting question randomly appearing in my Twitter feed:
If you had a greenfield network, would you choose SR-MPLS, or SRv6? And why?
TL&DR: SR-MPLS, assuming you’re building a network providing end-to-end connectivity between hardware edge devices.
Now for the why part of the question:
- SR-MPLS is just a different (better) control plane for the ancient MPLS data plane. All you need to use it on existing hardware is a software update. SRv6 is a brand-new data plane technology and might not be available in the gear you’re interested in.
- SR-MPLS is still simple label swapping. You might need longer label stacks, but that’s it. SRv6 (assuming we can agree which version of it we want to use – there’s also SRm6) is way more complex to implement in hardware, which might result in more expensive hardware.
- The overhead of SR-MPLS is just the MPLS label stack. The overhead of SRv6 is a set of IPv6 extension headers plus a brand new IPv6 header in case the payload is not an IPv6 packet.
- There are tons of technologies riding on top of MPLS, including FRR/LFA, L3VPN, 6PE, and EVPN. Replacing LDP with SR-MPLS does not affect any of them (OK, LFA gets easier ;). Of course you can implement all of these technologies on top of SRv6, but that requires new code, which inevitably results in new bugs.
- If you believe in centralized controller-based networking (because that worked so well in Frame Relay and IBM SNA days), you can use a controller to push the SR-MPLS label stack or SRv6 extension header chain into network nodes.
But wait, it gets worse. Contrary to the popular PowerPoint-based wisdom:
- SRv6 is no more secure than MPLS.
- SRv6 running on top of public IPv6 network is therefore as secure as MPLS-over-GRE-over-IP.
- SRv6 does not enable service insertion any more than VXLAN, Geneve, Network Services Header (NSH), or MPLS.
A quick detour into the service insertion land:
- You can always implement service insertion with VLAN- or VRF stitching (even in VXLAN/EVPN world).
- Implementing any other type of service insertion requires cooperation between the network and the node providing the service1. That’s why VMware NSX requires tight integration with the software providing east-west traffic inspection, and why NSH never took off.
- Anything else is PowerPoint-based handwaving (although I would love to be proven wrong).
Does that mean that SRv6 makes absolutely no sense? Not exactly; if you want to push traffic through virtual machines or containers, it’s (conceptually) easier to use SRv62 than SR-MLPS as you don’t have to think about pulling LSPs into your virtual world3. As soon as you open that can of worms, you have to ask “wrong with VXLAN or Geneve?” or “what extra value does SRv6 bring?” unless you desperately want to have Segment Routing because it sounds cool.
-
The proof is left as an exercise for the reader ↩︎
-
Ignoring for the moment the pain of making sure containers have all the necessary Linux drivers, and the pain of provisioning all those extra tunnel interfaces before starting the containers. ↩︎
-
Hint: just because you could build SR-MPLS paths into containers or virtual machines doesn’t mean that you should. If you want to insert virtualized workloads into a forwarding path, use something that rides on top of IPv4/IPv6. ↩︎
I like SR-MPLS because it doesn’t switch labels on every hop, (assuming uniformly configured node SIDs) which makes debugging easier.
I used to think SRv6 was great, until I started looking at what was needed to prevent 3rd parties from sending traffic across weird paths in my network. Then I realised that having a separate layer like MPLS has its benefits :)
It’s hard to get traffic to VMs, but it’s also much harder for (customer?) VMs to mess with my network…
Many thanks for such an interesting post.
Do you think vendors current supported label max stack might be an issue when trying to route a packet from source using Adj-SIDs on relatively big sized (and meshed) cores? Many seem to be proposing to use SRv6 to overcome this...
... and I'd like to hear about hardware that has problems with MPLS label stack but can handle SRv6 extension headers.
Completely agree with that.
The problem with SR-MPLS is that it does not play well with commodity hardware and ECMP. The label stack depth renders many commodity ASIC's unable to make a ECMP decision to send the flow on. Entropy labels doesn't really help since each takes two-labels and, on some commodity hardware, would require one every 2 SR labels. So the solution is great if you want to continue to purchase big iron ASIC's for the middle of your network.
SRv6 offsets this via the entropy field. Allowing the ingress point to set a entropy value which mid-points in the network can hash on.
... and the problem with SRv6 is that it doesn't seem to be supported by commodity hardware at all.
Found a document claiming "25 hardware implementations", but never found a corresponding list with 25 entries (unless it's the Jericho chipset, which one could hardly call "commodity").
The thing with SR-MPLS is that most of the presentations and slideshows available tend to emphasize how SR-MPLS is based on a lot of stacked MPLS labels or "instructions", with which to steer the traffic throughout the network via desired path. But at least in our case the main driver for migrating from LDP and RSVP-TE to SR-MPLS was to simplify the network and as such most of our traffic is essentially forwarded with only one transport label, or temporarily with a few if TI-LFA is doing a local repair. It is beautifully simple, easy to understand and debug.
The traffic-engineering capabilities of SR-MPLS are great but I'd hope that people would also value much higher the simplicity that one can achieve with SR-MPLS. This also somehow relates to what Sander wrote above that it is so much fun when the label remains unchanged when the packet traverses through the network.
As a next step we are considering migrating from 6VPE to native IPv6 next-hops in order to get rid of all the IPv6 mapped IPv4 addresses.
I have no clear opinion on SR-mpls vs SRv6 yet. SR-mpls has smaller labels, less overhead. With IPv6 you have one name-space. Both are a benefit, but not decisive for me.
There is one aspect I don't see mentioned. Because most networks don't need it. Small(er) networks don't need it. Datacenter-networks don't need it. But if you are an ISP or a hyperscaler with a WAN network with several thousands of routers, this aspect becomes important.
When you have a thousand routers in your networks, you can put all of them in one (IS-IS) area. Maybe with 2k routers as well. But when you have several thousand routers, you want to use areas. If only to limit the blast-radius. But there are several technologies where you need to know the SID of the router at the endpoint where you want to send traffic. That means in SR-MPLS you need to advertise the /32 (or /128) of every router in your network. Plus their SID. If you do flex-algo, multiple SIDs even. That starts to add up to a lot of info in your LSPDB. Your L1L2 routers might need to advertise several thousands of prefixes down. This is a practical concern. You can do things like increase lsp-mtu. Or implement RFC5311. I don't find that an elegant solution.
The best thing would be to have summarizable SIDs. SR-MPLS doesn't have that. SRv6 does. So there SRv6 has an advantage, imho. But even when you have summarizable SIDs, you still need to know the mapping between /32 (or /128) and the SID. I don't think there's a good way to do that yet, besides advertising each individual /32 and SID combination.
I've been told a few times that "routing is a solved problem". I don't agree. Routing is maybe a solved problem for networks with <= 1000 routers. For larger networks, you still need to use a bunch of trick and kludges to make it work. We could use a better solution.