Todd Hoff (of the HighScalability fame) sent me a link to an interesting video describing load-balancing mechanisms used at Google and how they evolved over time.

If the rest of the blog post feels like Latin, you SHOULD watch the Load Balancing and Scale-Out Application Architecture webinar.

The beginning of the story resembles traditional enterprise solutions:

• Like everyone else, Google started with traditional hardware load balancers;
• When they introduced multiple data centers, they used different IP endpoints in each data center and DNS-based load balancing to send the clients to one of the data centers. It seems they used random DNS responses first, and slowly migrated to geolocation-based DNS load balancing;

Next they introduced a unified front-end system (GFE) that would send the requests to target web servers based on URLs (still pretty standard), and used health checks to detect failed servers, and persistent GFE-to-server sessions to reduce the setup latency.

The setup got a bit more interesting with the introduction of back-end global load balancing. The clients would connect to the local GFE, which might use a web server in another data center to serve the request.

Interestingly, this setup is faster than redirecting the client to the other data center, because it reduces the session (and TLS) setup latency (assuming RTT between the client and GFE is significantly lower than the RTT between data centers).

Eventually, Google needed more performance than could be offered with off-the-shelf hardware load balancers, and decided to implement their own load balancing solution. At the same time, they decided to fix the well-known DNS load balancing problems, and implemented “a method of sending clients to the closest data center without relying on DNS” – a fancy description of anycast ;).

It’s interesting how many large web properties use anycast for first-line load balancing – it seems the global Internet is stable enough to make this approach work for TCP sessions (UDP anycast was long used to implement scale-out DNS infrastructure).