TL&DR Summary: It depends on your business model

With the explosion of overlay virtual networking solutions (with every single reasonably-serious vendor having at least one) one might get the feeling that it doesn't make sense to build greenfield IaaS cloud networks with VLANs. As usual, there's significant difference between theory and practice.

You should always consider the business requirements before launching on a technology crusade. IaaS networking solutions are no exception.

If you plan to sell your services to customers with complex application stacks, overlay virtual networks make perfect sense. These customers usually need multiple internal networks and an appliance between their internal networks and the outside world. If you decide to implement the Internet-facing appliance with a VM-based solution, and all subnets behind the appliance with overlay virtual networks, you're almost done.

Customers buying a single VM, and maybe access to central MySQL or SQL Server database, are a totally different story. Having a subnet and a VM-based appliance for each customer paying for a single VM makes absolutely no sense. We need something similar to PVLANs, and the only overlay virtual networking product with a reasonably simple PVLAN implementation is VMware NSX for Multiple Hypervisors.

If you want to use any other hypervisor/virtual networking platform, you have to get creative:

• Use a single subnet (VLAN- or overlay-based) and protect individual customer VMs with VM NIC firewall (or iptables)

• When using an overlay-based subnet for numerous single-VM customers, use a simple L2 or L3 gateway to connect the subnet to the outside world. Most overlay solutions include hardware or software gateways, and a 2-NIC Linux VM will easily route 1Gbps of traffic with a single vCPU.
• Worst case, use small PVLANs. There's no need for large or stretched VLANs if every customer has a single VM, more so if you don't give the customers fixed IP addresses but force them to rely on DNS.

Need help?

Check out my virtualization webinars or get in touch if you need design review or technology recommendation.

The webinars to consider include:

• Cloud Computing Networking if you need a broad technology overview;
• Virtual Firewalls if you want to know more about appliance- and NIC-based virtual firewalls;
• Overlay Virtual Networking if you’re looking for in-depth architecture and product details;
• VMware NSX Architecture if you’re evaluating the feasibility of VMware NSX;
• VXLAN Technical Deep Dive if you plan to build your cloud with VXLAN.

Not sure which webinar to watch? Try yearly subscription.

But wait, there's more!

I will be talking about Software-Defined Data Centers and private cloud infrastructure @ Interop 2014 Las Vegas. See you there!