Packet forwarding behavior of VMware NSX and Hyper-V Network Virtualization is well documented; no such documentation exists for Amazon VPC. However, even though Amazon uses a proprietary solution (heavily modified Xen hypervisor with homemade virtual switch), it’s pretty easy to figure out the basics from the observed network behavior and extensive user documentation.
More than 15 years ago the cover story of ACM netWorker magazine discussed the dawn of the stupid network – an architecture with smart edge nodes and simple packet forwarding code. Obviously we learned nothing in all those years – we’re still having the same discussions.
Here are a few juicy quotes from that article (taken completely out of context solely for your enjoyment).
Gateways between overlay virtual world and (VLAN-based) physical reality are a crucial component in every design using overlay virtual networks. Ideally one could use virtual appliances, but sometimes the users keep asking for layer-2 gateways.
Initial release of Hyper-V Network Virtualization (HNV) was an add-on to the Hyper-V Extensible Switch, resulting in an interesting mixture of bridging and routing. In Windows Server 2012 R2 the two components became tightly integrated, resulting in a pure layer-3 solution.
Every time I talk about small (per-application) virtual appliances, someone inevitably cries “And who will manage thousands of appliances?” Guess what – I’ve heard similar cries from the mainframe engineers when we started introducing Windows and Unix servers. In the meantime, some sysadmins manage more than 10.000 servers, and we’re still discussing the “benefits” of humongous monolithic firewalls.
Once you decide to use BGP as the routing protocol in your DMVPN network, you face a few more design choices:
- Should you use IBGP or EBGP?
- Should you use a unique AS number for every DMVPN site, or the same AS number on all spoke sites?
Last week I explained how layer-2 and layer-3 packet forwarding works in VMware NSX – a solution that closely emulates traditional L2 and L3 networks. Hyper-V Network Virtualization (HNV) is different – it’s almost a layer-3-only solution with only a few ties to layer-2.
I had three SDN 101 presentations during last week’s visit to South Africa and had tried really hard to overcome my grumpy skeptic self and find the essence of SDN while preparing for them. As I’ve been thinking about controllers, central visibility and network device programmability, it struck me: we already had SDN in 1993.
In the first Terastream blog post I mentioned Deutsche Telekom decided to use an IPv6-only access network. Does that mean they decided to go down the T-Mobile route and deployed NAT64 + 464XLAT? That combo wouldn’t work well for them, and they couldn’t use MAP-E due to lack of IP address space, so they deployed yet another translation mechanism – Lightweight 4over6.
The easiest way of connecting overlay virtual networks implemented with VMware NSX for vSphere to the outside world is NSX Edge Services Router. It’s a much improved version of vShield Edge and provides way more than just layer-3 forwarding services – it’s also a firewall, load balancer, DHCP server, DNS forwarder, NAT and VPN termination device.
Dan sent me the following question:
I had another read of the ‘Building IPv6 Service Provider Networks’ material and can see the PE routers use site local ipv6 addressing. I’m about to build another small service provider setup and wondered: would you actually use site local for PE loopbacks etc, or would you use ULA or global addressing? I’m thinking ULA would be better from a security point of view?
TR&DR summary: Don’t do that.
Even though I questioned the wisdom of writing your own network programming applications, I know I would immediately jump into those waters if I were 20 years younger. If you’re like my younger self, you might want to keep a few guidelines in mind.
Months ago VMware started promoting NSX with a catchy “fact” – you can deploy a new VM or virtual disk in minutes, but it usually takes days or more before you can get a new VLAN or a firewall or load balancer rule from the networking team.
Ignoring the complexity of network virtualization, they had a point, and the network services rigidity really bothered me … until I finally realized that we’re dealing with a broken process.
All overlay virtual networking solutions look similar from far away: many provide layer-2 segments, most of them have some sort of distributed layer-3 forwarding, gateways to physical world are ubiquitous, and you might find security features in some products.
The implementation details (usually hidden behind the scenes) vary widely, and I’ll try to document at least some of them in a series of blog posts, starting with VMware NSX.
Almost a year ago rumors started circulating about a Deutsche Telekom pilot network utilizing some crazy new optic technology. In spring I’ve heard about them using NFV and Tail-f NCS for service provisioning … but it took a few more months till we got the first glimpses into their architecture.
TL&DR summary: Good design always beats bleeding-edge technologies