You probably know the old saying – if the mountain doesn’t want to come to you, you have to go out there and climb it. vCider, a brand-new startup launching their product at Gigaom Structure Launchpad, decided to do something similar in the server virtualization (Infrastructure-as-a-Service; IaaS) space – its software allows IaaS customers to build their own virtual layer-2 networks (let’s call then vSubnets) on top of IaaS provider’s IP infrastructure; you can even build a vSubnets between VMs running within your enterprise network (private cloud in the cloudy lingo) and those running within Amazon EC2 or Rackspace.

Full disclosure: Chris Marino from vCider got in touch with me in early June. I found the idea interesting, he helped me understand their product (even offered a test run, but I chose to trust the technical information available on their web site and passed to me in e-mails and phone calls), and I decided to write about it. That’s it.

Before you get too ecstatic – they did not solve IaaS scalability issues (but they’re definitely looking in the right direction). Their software creates virtual Ethernet devices (transporting MAC frames over UDP) within the virtual machines. To use it, you have to install their device driver and control daemon in your VM OS (Linux only at the moment).

Next hurdle: every dynamic overlay implementation (think DMVPN) needs a registration and mapping service. vCider implements the VM registration and MAC-to-IP mapping process as a web service (your servers have to connect to it to register their IP and MAC addresses), making vCider client configuration totally dependent on the availability of vCider web site. If their site goes down, your vSubnet will continue to work ... until it encounters a configuration change (including physical IP address change).

Last but definitely not least – anyone implementing MAC-over-IP service has to deal with flooding (broadcasts, multicasts, unknowns). vCider decided to go the brute-force way and do packet replication at source (honestly, you can’t do much more over a generic IP infrastructure). They are already talking about IP multicast implementation and one could hope for ARP caches in some distant future.

More technical details

Configuration through vCider’s web site: VM clients access vCider’s web site only during the registration process and after a configuration change. The current MAC-to-IP mappings are stored in the vCider software installed in the VM; your vCider-based vSubnet can continue to operate even if their web site is unreachable as long as your configuration (including VM’s physical IP address) doesn’t change.

Physical IP address changes: If the IP of your VM can change, the VM has to re-register with vCider’s web site. However, they don’t identify VMs participating in a vSubnet by their IP addresses but by internally-generated node IDs. The registration of new MAC-to-IP mapping is thus automatic and requires no manual intervention.

Disk cloning. Even though node ID is stored on the VM’s disk image, you can clone the disk and start another VM with it. The vCider configuration software will realize you just created a duplicate node ID, reject the registration, create new node ID for the newly-launched VM and wait for configuration instructions.

NAT: In theory, their architecture could allow them to traverse NAT, but this functionality hasn’t been implemented yet.

IPv6: No support for IPv6 transport at the moment (obviously you can run IPv6, IPX, DECnet, OSI or anything else on top of their virtual Ethernet driver)

Encryption: By default, all traffic is encrypted with AES using per-VM shared keys (each VM has its own encryption key propagated to all other VMs in the same vSubnet).

Where could I use vCider?

vCider could help you solve your inter-VM connectivity issues if you’re an organization deploying VMs in more than one cloud.

It’s not the best solution for the “we’ll do whatever stupidity it takes to make us transparent” IaaS builders, as it requires intra-VM device driver.

It could, though, help creative IaaS builders that think like Amazon (we’ll serve 95% of customers, who cares if someone wants to run Windows NLB cluster in a cloud) – you put customer VMs in shared L3 subnets, protect them with a firewall that allows inbound SSH and UDP port 8002 (vCider) traffic and let the customers build their own networks.

Pros

It’s cool. And it’s MAC-over-IP, so I have to like it.

Also, it makes cloudbursting (yeah, I’m catching up on the buzzword bingo) easier. You could deploy your VMs anywhere, make sure you have vCider running on them, and they could instantaneously talk to each other without further configuration. Definitely easier than building site-to-site VPNs with IPsec.

You could move your VMs around, start them within a different cloud and they could still talk to each other. Multi-vendor public cloud infrastructure controlled through a web service. How cool is that?

Cons

I understand the ease-of-provisioning benefits, but honestly, what’s the use case for L2 network (OK, I know it’s easier to implement than L3 forwarding) across unpredictable WAN?

Connecting VMs running in different environments together makes sense, but the L2 connectivity will give server or apps people all sorts of crazy ideas (hint: HA clusters across cloudy skies). Prepare to spend even more time trying to persuade them the bandwidth fairy is just a myth (and pixie dust or OpenFlow does not reduce latency to zero) ... or you could let them crash big time (no, that’s not fair).

Oh, and while talking about ease-of-provisioning, why don’t we deploy a DHCP server in one of our VMs and let all others auto-number themselves?

Next problem: traffic trombones. They are bad enough in controlled virtualized environment and start getting out of hand when you deploy virtual networking appliances within a single data center. While vCider product itself does not introduce any extra tromboning (traffic is exchanged directly between VMs in the same vSubnet), imagine someone deploying Linux-based routers linking virtual networks willy-nilly across multiple cloud providers. Could you even start imagining where the traffic would actually flow (and how you’d troubleshoot that)?

One more: security. If you want vCider to work between VMs in your enterprise network and those running within public clouds, you have to drill a pretty big hole in your firewall (for the same reason many enterprises don’t allow SSL VPNs to be used from inside the network to third-party destinations). And don’t forget that your firewall won’t be able to filter/control the traffic flowing between those VMs.

Last one: while the developers will love vCider’s easy installation and web-based configuration, the operations people will go totally bonkers when faced with the idea that their network connectivity depends on a third-party mapping service. To get a real foothold in enterprise networks, vCider will have to start offering appliance-based solution that can be installed and controlled by the enterprise IT (there’s a very good reason Google is doing the same).