A while ago I wrote about performance bottlenecks of Open vSwitch. In the meantime, the OVS team drastically improved OVS performance resulting in something that Andy Hill called Ludicrous Speed at the latest OpenStack summit (slide deck, video).
The numbers quoted in the presentation were 72K flows (with the new default being 200K flows) and 260K pps.
200K flows is definitely more than enough to implement MAC/IP forwarding for 50 VMs (after all, that’s 4000 flows per VM), and probably still just fine even if you start doing reflexive ACLs with OVS (that’s how NSX MH implements pretty-stateful packet filters).
What I’m assuming these days is 50:1 VM packing ratio (and you can expect 200:1 or more for Docker containers) on a reasonably recent server with 500GB of RAM, a dozen of cores and two 10GE uplinks. YMMV.
On the other hand, 260K pps is just over a gigabit per second assuming an average packet size of 500 bytes (IMIX average is 340 bytes) or around 3 Gbps with 1500-byte packets.
To put this number in perspective: Palo Alto virtual firewall can do ~1 Gbps (while doing slightly more than packet forwarding, so it burns four vCPUs), and the venerable ancient vShield Edge 1.0 managed to get 3 Gbps of firewalled traffic through userland VM while burning a single core.
The blog post on Network Heresy indicates OVS can do much more than what the presentation mentions (after all, those numbers are from a production deployment and thus represent the characteristics of actual compute infrastructure and workload), but considering that the typical server I mentioned would have at least 2 10GE uplinks (which would result in 40 Gbps of marketing bandwidth), the 1-3 Gbps throughput looks awfully low – maybe it's just that the production workloads described in the presentation don't need more than that, in which case we might not have a problem at all.
Another data point
I found another data point while researching the performance changes in recent OVS releases: an OpenStack Wiki article lists ipref speed between two Linux hosts running on different hypervisors using OVS @ ~1.4 Gbps. I was able to get 10 Gbps out of ipref running on Linux hosts on top of vSphere 4.x (on UCS blades) years ago. Honestly, I'm a bit confused.
Have I missed anything? Please share your opinions in the comments.