Category: Scalability

One of my readers sent me this question:

I have an Internet edge setup with two routers connected to two upstream ISPs and receiving full BGP routing table from them. I’m running iBGP between my Internet routers. Is there a formula to estimate convergence time if one of my uplinks fail? How many updates will I need to get the entire 512K routes in BGP table and also how much time it would take?

As always, the answer is it depends.

A week or so ago I described why a properly implemented hypervisor-based overlay virtual networking data plane is not a scalability challenge; even though the performance might decrease slightly as the total number of forwarding entries grow, modern implementations easily saturate 10GE server uplinks.

Scalability of the central controller or orchestration system is a totally different can of worms. As I explained in the Scaling Overlay Networks, the only approach that avoids single failure domain and guarantees scalability is scale-out control plane architecture.

In the Myths That Refuse to Die: Scalability of Overlay Virtual Networking blog post I wrote “number of MAC addresses has absolutely no impact on the forwarding performance until the MAC hash table overflows”, which happens to be almost true.

If you watched the Network Field Day videos, you might have noticed an interesting (somewhat one-sided) argument I had with Sunay Tripathi, CTO and co-founder of Pluribus Networks (start watching at around 32:00 to get the context). Let’s try to get the record straight.

“Thou Shalt Have No Chokepoints” is one of those simple scalability rules that are pretty hard to implement in real-life products. In the Distributed Data Plane part of Scaling Overlay Networks webinar I listed data plane components that can be easily distributed (layer-2 and layer-3 switching), some that are harder to implement but still doable (firewalling) and a few that are close to mission-impossible (NAT and load balancing).

Every major hypervisor and networking vendor has an overlay virtual networking solution. Obviously they’re not identical, and some of them work better than others in large-scale environments – an interesting challenge we tried to address in the Scaling Overlay Virtual Networks webinar. As always, we started by identifying the potential problems.

The edited videos for Scaling Overlay Virtual Networking webinar are available on ipSpace.net Content site. Nuage Networks sponsored the webinar; the videos are thus publicly available (without registration).

Stumbled upon a hilarious description of challenges encountered when trying to scale distributed systems (cluster of controllers running centralized control plane comes to mind).

It starts with “If someone tells you that scaling out a distributed system is easy they are either lying or drunk, and possibly both,” and gets better and better. Enjoy!

A while ago I had an interesting discussion with a fellow SDN explorer, in which I came to a conclusion that it makes no sense to insert an overlay virtual networking SDN controller between cloud orchestration system and virtual switches. As always, I missed an important piece of the puzzle: federation of cloud instances.

2014-11-04 16:48Z: CJ Williams sent me an email with information on SDN controller in upcoming Windows Server release. Thank you!

The latest release of Cisco Nexus 1000V for vSphere can handle twice as many vSphere hosts as the previous one (250 instead of 128). Cisco probably did a lot of code polishing to improve Nexus 1000V scalability, but I’m positive most of the improvement comes from interesting architectural changes.