… updated on Wednesday, February 1, 2023 07:49 UTC
What Is Anycast?
Whenever I start digging into technical details, I learn something new. A few days ago I’ve stumbled across the term anycast, which is a very interesting way to solve scalability issues:
- Deploy geographically dispersed servers using the same IP address (obviously they would also need a unique IP address or you wouldn't be able to manage them);
- Advertise your service as residing on that IP address (for example, use the IP address in NS records for DNS zones you host)
- Advertise the IP address (or corresponding IP prefix) into the Internet from multiple locations.
Anycast works best with simple request-response UDP applications (for example, DNS) and has been used for ages to implement large-scale DNS deployments (for example, root name servers).
Using anycast for TCP sessions is trickier. It works reasonably well in the global Internet where the anycast endpoints are far enough from each other and true equal-cost multipathing is rare. CloudFlare is using anycast globally and managed to tweak their TCP/IP stack to use anycast source IP addresses. LinkedIn has fewer points-of-presence and decided to use regional anycast.
Using anycast TCP endpoints in data centers is harder and requires fine-tuned interaction with network-layer multipathing as well as a bag of intricate tricks.
For more details, watch the Load Balancing and Scale-Out Architectures part of the Data Center Infrastructure for Networking Engineers webinar and the Global Accelerator video in the Amazon Web Services Networking webinar.
Revision History
- 2023-02-01
- Added links to UDP and TCP anycast use cases
A couple of papers on anycast + tcp:
http://www.nanog.org/mtg-0606/pdf/matt.levine.pdf
http://www.netcat.ca/tag/tcp-anycast/
It is true that if the node goes down that the tcp sessions would fail, however, that is the case if you're using anycast or not.
The best way to do anycasting is to run a routing daemon on the host, so as to minimize the risk of blackholing traffic in the event of a host outage.
* Your upstream ISP has to provide effective BGP communities (#4)
* Coasts and countries appear to provide enough padding for route instability (#6)
* Nodes that are near by could possibly require state between each node if routes are unstable (#6)
* There's also the "Gotchas" slide (#20)
However, the actual statistical data they've provided (#19) indicate that the failure rate is so low that you can ignore it, meaning that although my reservations were valid, they were mostly academic :) Thanks for pointing that out.
http://www.cisco.com/en/US/docs/ios/solutions_docs/ip_multicast/White_papers/anycast.pdf
It's using MSDP to make Anycast RP possible.
Anycast is also commonly used for DNS and RADIUS. For an introduction on DNS Anycast see http://www.gdt.id.au/~gdt/presentations/2006-07-18-linuxsa-anycast/