Stefan de Kooter (@sdktr) sent me a follow-up question to my Going All Virtual with Virtual WAN Edge Routers blog post:

How would one interface with external Internet in this scenario? I totally get the virtual network assets mantra, but even a virtual BGP router would need to get a physical interconnect one way or another.

As always, there are plenty of solutions depending on your security needs.

Setting the mindset

Please read the Going All Virtual with WAN Edge Routers blog post before continuing, and keep in mind that most private cloud deployments don’t need more than two ToR switches connecting dozens of high-density servers. This blog post is focusing on reasonably sized Greenfield private cloud scenario (and your mileage may vary).

Connect the Internet links to your ToR data center switches

This design is the easiest to implement: don’t use WAN edge routers, plug the Internet uplinks straight into a dedicated VRF configured on the ToR switches, run BGP on the ToR switches, and do the basic traffic scrubbing with an inbound ACL.

Potential drawbacks:

• You can’t get anything close to full Internet routing table into the forwarding hardware of a ToR switch;
• Some data center switches don’t run BGP at all, others have limited set of BGP features available;
• Internet (outside) and inside segments are on the same switch – but then many organizations use the same approach with physical firewalls dual-connected to the same switch.

Want to know how many IP forwarding entries you get on various ToR and core switches? Check out the Data Center Fabric Architectures webinar.

Use the ToR switch to connect Internet link to an outside VLAN

Like in the previous design, plug the Internet uplink into a port on a ToR switch, but don’t run that port in a VRF, but in a L2-only VLAN that extends to the hypervisor host with the virtual router.

Advantages (as compared to the previous design):

• ToR switch does not run IP in the Internet VLAN, so it’s impossible to crash it with a malformed IP packet;
• An attacker cannot send traffic to the control plane of the ToR switch (all L3 traffic is forwarded to hosts/VMs in the same VLAN).

Use a dedicated interface on the hypervisor host

In the previous design we used a trunk interface to bring the Internet VLAN to the hypervisor host (similar to one-armed firewall design). We could enforce stricter traffic separation by using a dedicated hypervisor host interface for the outside segment, and use a dedicated virtual switch within the host to link the virtual router (or firewall) to that interface.

This design does prevent accidental inter-segment traffic leaking due to fat fingers (or misconfigured promiscuous mode) but does not increase protection against hypervisor bugs, as all virtual switches share the same code base (and sometimes even the same data structures).

If an intruder manages to crash the hypervisor host with a malformed packet, you’ll be dead in the water regardless of how many virtual switches you use. On the other hand, I haven’t heard of a single exploit that would crash the hypervisor by sending a crafted packet to a VM, whereas there’ve been plenty of cases where a router would crash when served a properly malformed packet.

Connect the Internet uplink straight into a hypervisor host

If your security policy dictates a strict separation between outside and inside segments (example: you’d use multiple physical switches for different segments in a DMZ), plug the Internet uplink straight into a spare interface of the hypervisor host.

In this design, the Internet traffic never reaches the ToR switch, so it’s marginally safer than previous designs, but I think we’re already deep into the paranoia land at this point.

Connect the Internet uplink straight to the virtual router NIC

Finally, you could use hypervisor bypass to connect the virtual router straight to the physical NIC (or a Virtual Function of a SR-IOV-capable NIC).

In both cases the virtual router has to support the physical NIC (= it has to have the proper device drivers), which might be a challenge if the vendor relies on using VMXNET3 or virtio drivers. The only exceptions (that I know of) are:

• Cisco’s Palo ASIC which looks exactly like VMXNET3 device when used in VM-FEX mode (that’s why you can use vMotion with VMDirectPath);
• Snabb switch use very smart tricks to presents SR-IOV Virtual Function of an Intel NIC as virtio device to the VM.

Need more information?

You’ll find hours of additional content in cloud and virtualization resources on ipSpace.net – all of it available with the yearly subscription (you can also buy individual webinars or whole roadmaps).