Meeting Brad Hedlund in person was definitely one of the highlights of my Interop 2013 week. We had an awesome conversation and quickly realized how closely aligned our views of VLANs, overlay networks and virtual appliances are.
Not surprisingly, Brad quickly improved my ideas with a radical proposal: running BGP between the virtual and the physical world.
Let’s revisit the application stack I used in the disaster recovery with virtual appliances post. One of the points connecting the virtual application stack with the physical world was the outside IP address of the firewall (or load balancer if you’re using bump-in-the-wire firewall).
Now imagine inserting a router between the firewall and the outside world, allocating a prefix to the application stack (it could be a single /32 IPv4 prefix, a single /64 IPv6 prefix, or something larger), and advertising that prefix from the virtual router to the physical world via BGP.
Before you start writing a comment complaining how three virtual appliances in sequence reduce performance and introduce unnecessary network traversals: as most virtual appliances these days use Linux, it isn’t that hard to add a few more daemons to the same VM. The three boxes in the picture could be a single VM if you prefer performance optimization over flexibility.
You could easily preconfigure the ToR switches (or core switches – depending on your data center design) with BGP peer templates, allowing them to accept BGP connections from a range of directly connected IP addresses, assign outside IP address to the virtual routers via DHCP (potentially running on the same ToR switch), and use MD5 authentication to provide some baseline security.
An even better solution would be a central BGP route server where you could do some serious authentication and route filtering. Also, you could anycast the same IP address in multiple data centers, making it easier for the edge virtual router to find its BGP neighbor even after the whole application stack has been migrated to a different location.
This twist on the original idea makes the virtual application stack totally portable between compatible infrastructures. It doesn’t matter what VLAN the target data center is using, it doesn’t matter what IP subnet is configured on that VLAN, when you move the application stack the client-facing router gets an outside address, establishes a BGP session with someone, and starts advertising the public-facing address range of the application.
I described the basics of overlay networks in Cloud Computing Networking webinar and in Overlay Virtual Networking Explained presentation @ Interop 2013. In-depth overlay networking concepts will be explained in Data Center Network Virtualization and Overlay Virtual Networking webinars in fall 2013.