Does anyone know what secret networking magic the Cloud providers are doing deep in their fabrics which are not exposed to consumers of their services?
TL&DR: Of course not… and I’m guessing it would be pretty expensive if I knew and told you.
- They must be using overlay virtual networking to implement virtual networks. Nothing else would scale to what they need – scalability numbers achieved by products like Cisco ACI are laughable from a hyperscaler perspective.
- It must be either complex enough or large enough not to be implementable on ToR switches.
- AWS is the only one of the big three to offer bare-metal servers, and we know their magic runs in their smart NICs (as Pensando so proudly points out like it would validate their business model). Azure seems to be using FPGAs, and Google relied on a software solution.
For more details see:
- Azure accelerated networking
- Andromeda: performance, isolation, and velocity at scale in cloud network virtualization
Network load balancing and Internet-facing NAT are truly interesting. Microsoft wrote a paper describing an early implementation of their Network Load Balancer, and it’s reasonably easy to envision how the same approach could be used for NAT. I’m positive AWS is doing something similar.
- Maglev: A Fast and Reliable Software Network Load Balancer
- Stateless datacenter load-balancing with Beamer
While you could solve load balancing with a proper combination of worker nodes and hypervisor tricks, I’m positive other complex networking services like AWS Transit Gateway run on top of the virtual networking (like virtual machines), but in multi-tenant bare-metal instances. For an overview of this idea, see Real Virtual Routers used in Oracle Cloud.
It seems like most everything else runs in managed VMs. It’s pretty obvious Azure application load balancing is implemented with virtual machines and a Network Load Balancer sitting in front of them, VPN gateways are supposedly Windows servers (that’s why it took 30 minutes to provision one), and even their recently introduced Route Server is just two managed VMs, probably with somewhat-privileged access to the orchestration system. AWS and Google are probably using similar approaches, or they could be using multi-tenant bare metal servers for efficiency reasons… but do you really care about implementation costs if you charge them to the customer?
Anything else? Would appreciate comments with links to insightful papers.