1000 VM per Rack Is Perfectly Realistic

Wednesday, November 4, 2015 13:31 +0100

1000 VM per Rack Is Perfectly Realistic

Last year I claimed that you don’t need more than two switches in your data center (I’ll run a presentation on the same topic in a few days), but focused exclusively on the networking side of the equation.

Iwan Rahabok recently published a great blog post describing the compute- and storage parts of it. His conclusion: 1000 VM per rack is perfectly realistic.

Recent posts in the same categories

8 comments:

Iwan 1005 22-03 05 November 2015 07:29

Thanks Ivan for the post. Indeed there are many customers who could not cross the mental bridge that it is doable. It kinda reminds me of the 4-minute mile barrier :-)
Have a great day!
e1

Tom Carter 06 November 2015 04:07

Ivan,

I'm disappointed that I don't see any mention in that post about my primary concern for those types of racks: power and cooling. AnandTech suggests the Xeon 2699v3 uses around 175 watts (TDP is 145 watts). That means that, just for CPUs, rack power is 19.6 kilowatts, when you throw in motherboards, RAM, drives, etc, 30 kilowatts for a rack doesn't seem out of the question, especially factoring in two switches. 30kw is not un-doable in this day and age, but caveat emptor if you buy this and find out your datacenter only budgeted for 8kw per rack.

Replies

Ivan Pepelnjak 06 November 2015 13:37

Tom,

Thanks for the comment. I know I'm always missing something, and you just pointed out yet another caveat. Let me explore this a bit further.

Kind regards,
Ivan

Iwan 'e1' Rahabok 07 November 2015 00:10

Spot on! Thank you Tom for the correction. How did I miss that! I've updated my blog with an acknowledgement to your correction (and thank you for that), and link back to this post so readers can see your comment.

Your comment does echo what I always tell fellow virtualisation engineer. The physical stuff matters.

Unknown 09 November 2015 18:54

It becomes an even bigger concern if you are factoring co-lo into your international footprint. We were designing regional data hubs for a large enterprise that required very dense VM footprint with a minimal network edge. It wasn't 1,000 VMs per rack, but it did require 30kw. In the US, we found that most of the big shops would accommodate us. In EMEA and APAC... our selection was extremely small and in some cases it was treated as a custom job and priced accordingly. We even used a co-lo broker to help us locate willing providers. (This was a few years ago, it might be better now.)

As an early adopter of vBlock, we can concur with Ivan's point: you can build a very dense network with a surprisingly small footprint, especially if you're willing to leverage VM-based ADCs/Firewalls/Routers

Anonymous 21 November 2015 10:26

To autarch01
Please try very flexible operator in baltics www.datainn.lt/en. Great service and network www.baltichighway.com

LS 02 December 2015 19:11

1000? you're thinking too small :-)
I work for a cloud provider, we shove 250 VMs per HV, and 40 HVs per rack. Try 10,000 VMs per rack :-)

Replies

Ivan Pepelnjak 02 December 2015 20:33

A typical enterprise virtualization engineer would faint at what you're suggesting ;) But yeah, I agree with you - those numbers are definitely reachable if you're pushing the envelope (and let me guess: most of the VMs probably have 4GB of RAM or so, right?)

Add comment