Category: virtualization

Recently I was discussing the benefits and drawbacks of virtual appliances, software-defined data centers, and self-service approach to application deployment with a group of extremely smart networking engineers.

After the usual set of objections, someone said “but if we won’t become more flexible, the developers will simply go to Amazon. In fact, they already use Amazon Web Services.”

After the Designing Private Cloud Infrastructure workshop I had in Slovenia last week (in a packed room of ~60 people), someone approached me with a simple question: “I like the idea of using overlay virtual networks in my private cloud, but where do I start?”

I wrote (and spoke) at length about layer-2 and layer-3 gateways between VLANs and overlay virtual networks, but I still get questions along the lines of “how will you connect legacy servers to the new cloud infrastructure that uses VXLAN?”

A networking engineer was trying to persuade me of importance of hardware VXLAN VTEPs. We quickly agreed physical-to-virtual gateways are the primary use case, and he tried to illustrate his point by saying “Imagine you have 1000 servers in your data center and you manage to virtualize 80% of them. How will you connect them to the other 200?” to which I replied, “That doesn’t make any sense.” Here’s why.

Not sure I published a link to this video: the overview of VMware NSX Architecture (for additional details watch other videos from the VMware NSX Architecture webinar).

Friday roundtables are one of the best parts of the Troopers conference – this year we were busy discussing (among other things) how safe the hypervisors are as compared to more traditional network isolation paradigms.

TL&DR summary: If someone manages to break into your virtualized infrastructure, he’ll probably find easier ways to hop around than hypervisor exploits.

A comment by Pieter E. Smit on my vSphere Does Not Need LAG Bandaids post opened yet another can of worms: vSphere behavior on uplink recovery.

Short summary: vSphere starts using an uplink as soon as its physical layer becomes operational, which might happen during ToR switch startup phase, or before a ToR switch port enters forwarding state.

Marcos Hernandez sent me a nice list of updates/errata after watching the NSX firewalls video from the VMware NSX Architecture webinar:

If you’re building a Greenfield private cloud, you SHOULD consider using virtual network services appliances (firewalls, load balancers, IPS/IDS systems), removing the need for additional hard-to-scale hardware devices. But can we go a step further? Can we replace all networking hardware with x86 servers and virtual appliances?

I hope it’s obvious to everyone by now that flow-based forwarding doesn’t work well in existing hardware. Switches designed for large number of flow-like forwarding entries (NEC ProgrammableFlow switches, Enterasys data center switches and a few others) might be an exception, but even they can’t cope with the tremendous flow update rate required by reactive flow setup ideas.

One would expect virtual switches to fare better. Unfortunately that doesn’t seem to be the case.

I was running fantastic Network Security in a Private Cloud workshops in early 2010s and a lot of the discussions centered on the mission-impossible task of securing existing underdocumented applications, rigidity of networking team and their firewall rules and similar well-known topics.

The make all firewalls virtual and owned by the application team idea also encountered the expected resistance, but enabled us to start thinking in more generic terms.

Traditional firewalls are well-known chokepoints in any virtualized environment. The firewalling functionality can be distributed across VM NICs, but some of those implementations still rely on VM-based packet processing resulting in a local (instead of a global) performance bottleneck.

VMware NSX solves that challenge with two mechanisms: OpenFlow-based stateful(ish) ACLs in VMware NSX for multiple hypervisors and distributed in-kernel stateful firewall in VMware NSX for vSphere. You’ll find more details in the NSX Firewalls video recorded during the VMware NSX Architecture webinar.

Every time I plug a new device into my Windows laptop and it automatically discovers the device type, installs the driver, configures the devices, and tells me it’s ready for use, I wonder why we can’t have get the same level of automation in networking.

Consider, for example, a well-known vSphere link failover issue: if you forget to enable portfast on server-facing switch ports, some VMs lose connectivity for up to 30 seconds every time a switch reloads.

The layer-3-only Hyper-V Network Virtualization forwarding model implemented in Windows Server 2012 R2 thoroughly confuses engineers used to deal with traditional layer-2 subnets connected via layer-3 switches.

As always, it helps to take a few steps back, focus on the principles, and the “unexpected” behavior becomes crystal clear.

2014-02-05: HNV routing details updated based on feedback from Praveen Balasubramanian. Thank you!

If you want to implement overlay virtual networking with VMware products today, you have two options: use vCNS 5.5 or NSX for vSphere… and I would be hard pressed to choose one or the other.