Category: virtualization

A comment by Pieter E. Smit on my vSphere Does Not Need LAG Bandaids post opened yet another can of worms: vSphere behavior on uplink recovery.

Short summary: vSphere starts using an uplink as soon as its physical layer becomes operational, which might happen during ToR switch startup phase, or before a ToR switch port enters forwarding state.

Marcos Hernandez sent me a nice list of updates/errata after watching the NSX firewalls video from the VMware NSX Architecture webinar:

If you’re building a Greenfield private cloud, you SHOULD consider using virtual network services appliances (firewalls, load balancers, IPS/IDS systems), removing the need for additional hard-to-scale hardware devices. But can we go a step further? Can we replace all networking hardware with x86 servers and virtual appliances?

I hope it’s obvious to everyone by now that flow-based forwarding doesn’t work well in existing hardware. Switches designed for large number of flow-like forwarding entries (NEC ProgrammableFlow switches, Enterasys data center switches and a few others) might be an exception, but even they can’t cope with the tremendous flow update rate required by reactive flow setup ideas.

One would expect virtual switches to fare better. Unfortunately that doesn’t seem to be the case.

I was running fantastic Network Security in a Private Cloud workshops in early 2010s and a lot of the discussions centered on the mission-impossible task of securing existing underdocumented applications, rigidity of networking team and their firewall rules and similar well-known topics.

The make all firewalls virtual and owned by the application team idea also encountered the expected resistance, but enabled us to start thinking in more generic terms.

Traditional firewalls are well-known chokepoints in any virtualized environment. The firewalling functionality can be distributed across VM NICs, but some of those implementations still rely on VM-based packet processing resulting in a local (instead of a global) performance bottleneck.

VMware NSX solves that challenge with two mechanisms: OpenFlow-based stateful(ish) ACLs in VMware NSX for multiple hypervisors and distributed in-kernel stateful firewall in VMware NSX for vSphere. You’ll find more details in the NSX Firewalls video recorded during the VMware NSX Architecture webinar.

Every time I plug a new device into my Windows laptop and it automatically discovers the device type, installs the driver, configures the devices, and tells me it’s ready for use, I wonder why we can’t have get the same level of automation in networking.

Consider, for example, a well-known vSphere link failover issue: if you forget to enable portfast on server-facing switch ports, some VMs lose connectivity for up to 30 seconds every time a switch reloads.

The layer-3-only Hyper-V Network Virtualization forwarding model implemented in Windows Server 2012 R2 thoroughly confuses engineers used to deal with traditional layer-2 subnets connected via layer-3 switches.

As always, it helps to take a few steps back, focus on the principles, and the “unexpected” behavior becomes crystal clear.

2014-02-05: HNV routing details updated based on feedback from Praveen Balasubramanian. Thank you!

If you want to implement overlay virtual networking with VMware products today, you have two options: use vCNS 5.5 or NSX for vSphere… and I would be hard pressed to choose one or the other.

Chris Wahl claimed in one of his recent blog posts that vSphere doesn't need LAG band-aids. He's absolutely right – vSphere’s loop prevention logic alleviates the need for STP-blocked links, allowing you to use full server uplink bandwidth without the complexity of link aggregation. Now let’s consider the networking perspective.

During my ExpertExpress engagements with engineers building multi-tenant cloud infrastructure I often get questions along the lines of “How do I integrate my public IaaS cloud with my MPLS/VPN WAN?” Here are a few ideas.

Gordon sent me a whole list of NSX gateway questions:

• Do you need a virtual gateway for each VXLAN segment or can a gateway be the entry/exit point across multiple VXLAN segments?
• Can you setup multiple gateways and specify which VXLAN segments use each gateway?
• Can you cluster gateways together (Active/Active) or do you setup them up as Active/Standby?

The answers obviously depend on whether you’re deploying NSX for multiple hypervisors or NSX for vSphere. Let’s start with the former.

A few days ago I described how most OpenFlow data center fabric solutions use out-of-band control plane (separate control-plane network). Can we do something similar when running OpenFlow switch (example: Open vSwitch) in a hypervisor host?

TL&DR answer: Sure we can. Does it make sense? It depends.

Packet forwarding behavior of VMware NSX and Hyper-V Network Virtualization is well documented; no such documentation exists for Amazon VPC. However, even though Amazon uses a proprietary solution (heavily modified Xen hypervisor with homemade virtual switch), it’s pretty easy to figure out the basics from the observed network behavior and extensive user documentation.

Initial release of Hyper-V Network Virtualization (HNV) was an add-on to the Hyper-V Extensible Switch, resulting in an interesting mixture of bridging and routing. In Windows Server 2012 R2 the two components became tightly integrated, resulting in a pure layer-3 solution.