Category: design

Cloud builders are often using my ExpertExpress service to validate their designs. Tenant onboarding into a multi-tenant (private or public) cloud infrastructure is a common problem, and tenants frequently want to retain the existing network services appliances (firewalls and load balancers).

The Combine Physical and Virtual Appliances in a Private Cloud case study describes a typical solution that combines per-tenant virtual appliances with frontend physical appliances.

A few months ago I described how bandwidth limitations shatter the dreams of spread-out application stacks with elements residing (or being dynamically migrated) between data centers. Today let’s focus on bandwidth’s ugly cousin: latency.

TL&DR Summary: Spreading the server components of an application across multiple locations (multiple data centers or hybrid cloud deployments) can easily result in dismal performance even when there’s plenty of bandwidth available.

Facebook published their next-generation data center architecture a few weeks ago, resulting in the expected “revolutionary approach to data center fabrics” echoes from the industry press and blogosphere.

In reality, they did a great engineering job using an interesting twist on pretty traditional multi-stage leaf-and-spine (or folded Clos) architecture.

In his The Case for Hybrids blog post Mat Mathews described the Hotel California effect of public clouds as: “One of the most oft mentioned issues with public cloud is the difficulty in getting out.” Once you start relying on cloud provider APIs to provide DNS, load balancing, CDN, content hosting, security groups, and a plethora of other services, it’s impossible to get out.

Interestingly, the side effects of public cloud deployments extend into the realm of application programming, as I was surprised to find out during one of my Expert Express engagements.

It doesn’t make sense to build a new data center network to support legacy bare-metal server infrastructure. You’ll have to use relatively expensive 1G/10G ports to be able to connect the current and future servers, and once the server and virtualization engineers wake up and do hardware refresh you’ll end up with way too many ports (oh, and you do know that transceivers could cost more than the switching hardware, right?).

More than a year ago I wrote a response to a comment Pascal wrote on my Predicting the IPv6 BGP table size blog post. I recently rediscovered it and figured out that it’s (unfortunately) as relevant as it was almost 18 months ago.

Other people have realized we have this problem in the meantime, and are still being told to stop yammering because the problem is not real. Let’s see what happens in a few years.

Every time I’m running a classroom version of my Designing the Cloud Infrastructure workshop, I start with a simple question: “Who has more than 2000 VMs or bare-metal servers in the data center?”

I might see three hands on a good day; 90-95% of the audience have smaller data centers… and some of them get disappointed when I tell them they don’t need more than two ToR switches in their data center.

People talking about long-distance workload mobility and cloudbursting often forget the physical reality documented in the fallacies of distributed computing. Today we’ll focus on bandwidth, in a follow-up blog post we’ll deal with its ugly cousin latency.

TL&DR summary: If you plan to spread application components across the network without understanding their network requirements, you’ll get the results you deserve.

In July I wrote about an Amazon Kindle version of my Data Center Design Case Studies book and complained about their royalties model. Someone quickly pointed out how to adapt to their system: split the book into multiple volumes and charge $9.99 for each.

It took me months to get there, but the first two volumes are finally on Amazon:

A while ago Rick Parker told me about his amazing project: he started a meetup group that will build a reference private/hybrid cloud heavily relying on virtualized network services, and publish all documentation related to their effort, from high-level architecture to device and software configurations, and wiring plans.

In Episode 8 of Software Gone Wild Rick told us more about his project, and we simply couldn’t avoid a long list of topics including:

Building a private cloud infrastructure tends to be a cumbersome process: even if you do it right, you oft have to deal with four to six different components: orchestration system, hypervisors, servers, storage arrays, networking infrastructure, and network services appliances.

When I published the Data Center Design Case Studies book almost exactly a month ago, three chapters were still missing – but that was the only way to stop the procrastination and ensure I’ll write them (I’m trying to stick to published deadlines ;).

The first one of the missing chapters is already finished and available to subscribersand everyone who bought the book or Designing Private Cloud Infrastructure webinar (you’ll also get a mailing on Sunday to remind you to download the fresh copy of the PDF).

The Amazon Kindle version will be updated in a few days.

The last generations of high-end servers are amazing: they can have terabyte (or more) of RAM, dozens of CPU cores, and four (or more) 10GE uplinks. It’s easy to pack 100+ well-behaved VMs on a single server, reducing the whole data center into a private cloud that fits into a single rack.

If you’re an avid Kindle user, you can buy the Data Center Design Case Studies book on Amazon. Here’s the Amazon.com link, search should find it on most other Amazon marketplaces.

However, if you can survive reading the PDF version, please buy it straight off my web site. Here’s why:

Occasionally I get a question about some totally impossible implementation detail (example: can we use OpenStack OVS plugin on VMware to avoid buying NSX?). These questions are often coming from people who painted themselves into a corner and are now desperately looking for MacGyver’s shoelaces to pull themselves out.

It’s easy to blame the engineer who tries to do the obviously impossible, but it’s often not his fault – these days a lot of technical people get pulled into the game of Build a Cloud in Three Easy Steps.