Category:  IP routing

Moving a running VM into a foreign subnet is Mission Impossible due to stale ARP entries (anyone telling you otherwise is handwaving over a detail or two - maybe their VM doesn't communicate with other VMs in the same subnet), but it's entirely feasible to migrate a cold VM into a foreign subnet if you can fix IP routing. Here's how you can do the trick with Enterasys switches.

I spent the last few weeks blogging about the brave new overlay worlds. Time to return to VLAN-based physical reality and revisit one of the challenges of VM mobility: mobile IP addresses.

A while ago I speculated that you might solve inter-subnet VM mobility with Mobile ARP. While Mobile ARP isn’t the best idea ever invented it just might work reasonably well for environments with dozens (not millions) of virtual servers.

Enterasys decided to go down that route and implement host routing in their data center switches. For more details, watch the video from the Enterasys DCI webinar.

Routing protocols running on virtual appliances significantly increase the flexibility of virtual-to-physical network integration – you can easily move the whole application stack across subnets or data centers without changing the physical network configuration.

Major hypervisor vendors already support the concept: VMware NSX-T edge nodes can run BGP or OSPF¹, and Hyper-V gateways can run BGP. Like it or not, we’ll have to accept these solutions in the near future – here’s a quick survival guide.

VMware gave me early access to NSX hands-on lab a few days prior to VMworld 2013. The lab was meant to demonstrate the basics of NSX, from VXLAN encapsulation to cross-subnet flooding, but I quickly veered off the beaten path and started playing with routing protocols in NSX Edge appliances.

Enterasys implemented optimal layer-3 forwarding with an interesting trick: they support VRRP like any other switch vendor, but allow you to make all members of a VRRP group active forwarders regardless of their status.

Apart from a slightly more synchronized behavior, their implementation doesn’t differ much from Arista’s Virtual ARP, and thus shares the same design and deployment caveats.

For more information, watch the Fabric Routing video from the Enterasys Robust Data Center Interconnect Solutions webinar.

A while ago someone asked what the difference between access and prefix lists is on the Network Engineering Stack Exchange web site (a fantastic resource brought to life primarily by sheer persistence of Jeremy Stretch, who had to fight troves of naysayers with somewhat limited insight claiming everything one would want to discuss about networking falls under server administration web site).

The question triggered a lengthy wandering down the memory lane … and here's the history of how the two came into being (and why they are the way they are).

The Optimal L3 Forwarding with VARP/VRRP post generated numerous comments, ranging from technical questions about VARP (more about that in a few days) to remarks along the lines of “you can do that with X” or “vendor Y supports Z, which does the same thing.” It seems I’ve opened yet another can of worms, let’s try to tame and sort them.

I’ve blogged about the need for optimal L3 forwarding across the whole data center in 2012 when I introduced it as one of the interesting requirements in Data Center Fabrics webinar. Years later, the concept became one of the cornerstones of modern EVPN fabrics, but there are still only a few companies that can deliver this functionality in a more traditional environment.

I stumbled upon a fantastic RFC - Evolution of IP Model (RFC 6250) - that should be made mandatory reading for everyone remotely involved with networking. It describes numerous "truths" (politely called misconceptions) that everyone from programmers to network designers still rely upon. Some of my favorites: reachability is symmetric and transitive, loss is rare, addresses are stable, each host has a single interface and a single IP address ... Enjoy!

Adam Sweeney, VP of EOS Engineering @ Arista Networks posed me a challenging question after my I-so-hate-PBR-CLI rant: “Is there something in particular that makes the IOS PBR CLI so painful? Is there a PBR CLI provided by any of the other systems out there that you like a lot better?”

My Twitter friends helped me find the answer to the second question: PBR in Junos is even more convoluted than it is in Cisco IOS... but what would be a better CLI?

During one of my ExpertExpress consulting engagements I encountered an interesting challenge:

We have a network with two data centers (connected with a DCI link). How could we ensure the applications in a data center stay reachable even if all local Internet links fail?

On the face of it, the problem seems trivial; after all, you already have the DCI link in place, so what’s the big deal ... but we quickly figured out the problem is trickier than it seems.

Thomas wanted to check whether the IP traffic is actually delivered to a remote site and sent me the following question:

I would like to know whether the packets I sent from site A to site B have been received. I don't want to create test traffic using ip sla, I would like to know that the production traffic has been delivered. I could use ACL counters but I'm running a full mesh of tens of sites. Ipanema does this very well, but I'm surprised that this doesn’t exist on Cisco IOS.

Short answer: that’s not how Internet works.

One of my readers sent me a great question:

I agree with you that L2 DCI is like driving without a seat belt. But is L3 DCI safer in case of DCI link failure? Let's say you have your own AS and PI addresses in use. Your AS spans multiple sites and there are external BGP peers on each site. What happens if the L3 DCI breaks? How will that impact your services?

Simple answer: while L3 DCI is orders of magnitude safer than L2 DCI, it will eventually fail, and you have to plan for that.

A while ago someone sent me an interesting problem: the moment he enabled simple MPLS in his enterprise network with ip mpls interface configuration commands, numerous web applications stopped working. My first thought was “MTU problems” (the usual culprit), but path MTU discovery should have taken care of that.

Almost everyone agrees the current way of implementing virtual networks with dumb hypervisor switches and top-of-rack kludges (including Edge Virtual Bridging – EVB or 802.1Qbg – and 802.1BR) doesn’t scale. Most people working in the field (with the notable exception of some hardware vendors busy protecting their turfs in the NVO3 IETF working group) also agree virtual networks running as applications on top of IP fabric are the only reasonable way to go ... but that’s all they currently agree upon.