Anycast Fundamentals
I got into an interesting debate after I published the Anycast Works Just Fine with MPLS/LDP blog post, and after a while it turned out we have a slightly different understanding what anycast means. Time to fall back to a Wikipedia definition:
Anycast is a network addressing and routing methodology in which a single destination IP address is shared by devices (generally servers) in multiple locations. Routers direct packets addressed to this destination to the location nearest the sender, using their normal decision-making algorithms, typically the lowest number of BGP network hops.
Based on that definition, any transport technology that allows the same IP address or prefix to be announced from several locations supports anycast. To make it a bit more challenging, I would add “and if there are multiple paths to the anycast destination that could be used for multipath forwarding1, they should all be used”.
It’s also worth noting that outside of a single link-state area2, and in particular while looking at the routing or forwarding table, it’s often impossible to differentiate between an anycast prefix (prefix being advertised from multiple locations), redundantly connected site or autonomous system, or a topology that provides multiple equal-cost paths to the same destination. As is usually the case, the proof is left as an exercise for the reader.
With the terminology being out of the way, it’s time to turn to an excellent comment left by Minh Ha on the original blog post (the footnotes are mine).
While those who study transport mechanics in internetwork quickly realize the [anycast doesn’t work with traditional MPLS] claim is incorrect based on first principles, it’s always nice to see a conclusive proof laid out with step-by-step instructions and empirical data.
MPLS is Transport, and Transport is more concerned about addressing, routing, synchronization etc, while anycast, taken to its root, is a naming issue, so one can quickly realize there’s no fundamental reason that prevents MPLS from supporting anycast. Any limitation is strictly implementational, not technological. So it’s not too surprising how even old LDP code can take on anycast.
“MPLS is not a pure packet switching technology, but has a control plane based on virtual circuit switching.”3 … while I can see why Dmytro might have come to this conclusion, it’s not really true either. It’s true that MPLS is an evolution of ATM, and therefore, is not tunnel, but VC – never understand the pointless debates about whether MPLS is tunnel or VC as the people who claim it’s tunnel obviously don’t understand its history – but MPLS is more like loose VC. MPLS relies on a functional routing protocol to discover the paths, and LDP – if it can be called MPLS control plane as MPLS control plane can consist of way more than LDP – assigns the labels for the prefixes, so MPLS in an IP network is essentially packet-switching, not VC. If a core switch breaks down in a VC network, the VC traversing it breaks4, but in MPLS IP network, the LSP adjusts itself based on the underlying routing topology. LDP itself is a simplistic label assignment protocol, AFAIK it doesn’t perform any function central to a VC control plane, so all in all MPLS deviates a fair bit from traditional VC network.
But the most important part is: MPLS is more general than both VC and packet-switch, because it’s Transport, and this point is crucial. MPLS can therefore be generalized to support non-packet-switch network, in VC style or what not5. So all of these points make it obvious that nothing would stop MPLS from supporting anycast routing, or any form of routing, for that matter. And the reason this needs to be clearly identified is so vendors can’t use that as an excuse to sell SR for the wrong reason.
And I never understand what the excitement is with SR, given that it’s a product of SDN, itself a misconception(or a flight of fantasy) from the start6. SR is more like a point solution for a few special cases, and it tries to overcome the state problems by introducing more constructs into the network, like SRGB and global segments, that can lead to tight binding and confusion, esp. at large scale.
In fact Dmytro’s post already mentioned 3 deficiencies of SRGB mismatch. Introducing more moving parts, esp. global ones, into big distributed systems, always leads to complexity and unintended consequences. SR will run into problem with MPLS network where label is no longer just an abstract concept with no physical reality, but has to match the underlying resources. This ‘fitting the data’ requirement and SRGB can contradict each other, leading to much headache. And if one has to add a controller to SR to do TE, then one will run into all the scaling problems of centralized routing that SDN proponents have learnt the hard way.
-
… getting ECMP versus UCMP discussion out of the way ↩︎
-
… or level depending on your terminology ↩︎
-
Minh Ha is quoting the blog post that triggered my initial response. ↩︎
-
Slightly edited to make it easier to understand to audience not familiar with telco terminology. ↩︎
-
As I explained in other blog posts (and videos), SR-MPLS does make sense if you want to run MPLS transport as it simplifies the MPLS control plane (no IGP-LDP dependencies). SRv6 is obviously a different story. ↩︎
MPLS-T is more like classical ATM permanent VC. MPLS LDP is more like soft PVC in ATM. MPLS with RSVP/TE is more like PNNI in ATM. The control planes are very similar and somewhat re-invented only. The big difference is the dropping of SAR, since at gigabit seeds it does not have enough advantage on managing the jitter, like it is absolutely beneficial at kilobit speeds.
For me packet switching and VC are not contradictory. The term "virtual" denotes the usage of packet switching for simulating a TDM-style circuit behavior from the upper layers perspective. How on earth could you do VC without an underlying packet switching? In that sense ATM is also packet switching, just the packets are called "cells".
GMPLS can control classical physical, non-virtual circuits. Pure MPLS would not in the stricter term, if we do not limit it to the control plane, but we take it together with its data plane specifications.
Centralized SDN is also nothing new. Just the reincarnation of telephone switch management through the DCN. Plus using IP instead of X.25/OSI and some trendier fashionable protocols instead of the old ITU-T X.stuff. But the basic architecture is very similar. There are a lot of small networks where centralized SDN would not have a big problem. For example, the whole Hungarian telco network. Of course, this would be different in China... :-)
Anycast might be limited by routing policies. It is easy in a private network. On the public Internet, you should not originate other addresses than what you really own. Otherwise, the other ISPs might filter your advertisements as security issues. This could be also be a problem in a multi-AS, multi-provider MPLS network that might limit the anycast scoping to special peering agreements. You have a similar problem with mobile hosts or networks. It would not work by default in the public Internet.
Why is this important? Because with higher and higher interface speeds, per-packet processing at some point (if not already), proves too costly, and a switch to TDM or wavelength switching in the high-speed transport network is very desirable. In non-packet transport network, usually there's no label stack, so things like TE's facility bypass won't work. AFAIK, SRGB won't work either, esp. at boundary routers that do different styles of switching on different interfaces.
I strongly believe one of the reasons proponents uses to push SR is traffic engineering. But SR TE, if done using central controller, won't scale. Just an example: in photonic network, right now due to technological limitation, optical cross-connect can't do wavelength conversion, so much larger amount of TE info -- info on individual lambdas on every single link -- will have to be exchanged between TE routers, and more complex path computation will need to be applied. And signal attenuation and optical impairments due to effects of nonlinear optics such as stimulated Raman scattering or two-/four-wave mixing etc, also needs to be accounted for by path computation as well. Centralization won't scale.
Bela was right above when he said "There are a lot of small networks where centralized SDN would not have a big problem." Yes, centralization works fine for small scale, and fails spectacularly at large scale or as you start adding more features/info. So I can't see how SR is any better than existing TE, as the problem of TE is not something a label distribution method can resolve.
PNNI is a way to do QoS routing from the source, and it's a failure with scalability problem. Back then, route caching was proposed to solve scalability problem with PNNI, but route caching can lead to cache thrashing. Anything based on similar concept will fail at scale. Hierarchical destination-based, distributed routing is the only way to scale to very large size, because it distributes the states across the nodes, reducing both the computation burden at the endpoints and state explosion in the core.
As for LDP vs SR, since LDP is a minimalist protocol, I don't see much trouble. Protocol (and all software) should be kept simple as possible, and trying to stuff as much as one can into one (think BGP with 100k lines of code) is a bad idea. SR brings with itself much complexity, so I fail to see a tangible benefit. The ultimate reality check would be: have we got any large-scale outage caused by LDP? If not, LDP passes the simplicity-stability test.
Re Anycast being name, let me clarify. Address identifies where you are, whereas names identifies what/who you are. Address therefore, is location independent, and name, location-independent. So anycast address, by necessity, is a name. Anycast is but a name set with a rule to return one member of the set ( basically a special case of multicast ). The reason anycast and multicast are considered address is because the OSI model's addressing scheme is missing more than half its structure. If anycast is done right, it should be implemented at the application layer (Akamai got this right, though Idk if I get the fundamentals or just by sheer luck), and we won't have all of the issues Bela mentioned above (and the address exhaustion issue, and the RIB explosion issue etc). Port channel, by its nature, is also a form of anycast.