Redundant Data Center Internet Connectivity – High-Level Design

Yesterday I described the roadblocks you might encounter when faced with a seemingly simple challenge:

In a network with two data centers (connected with a DCI link), ensure the applications in a data center stay reachable even if its Internet links fail.

In the Solutions Corner (a brand new part of my web site) you’ll find a short high-level design document describing the overall solution and listing the technologies you could use to implement it (you might want to watch the video before reading the document).


  1. we've been running our main online presence across dual DC's (per region) for approx 10 years in just the fashion outlined in this design guide. our DC pairs are geographically relatively close together and we are lucky to have dark fiber between them which allows us have logically and physically separate connectivity for internal and external inter DC traffic. the external links directly connect our eBGP speakers in both DC's in a iBGP mesh.

    Additionally on the eBGP speakers in both DC's we advertise the local DC's prefix (say a /21) as well as a the larger /20 that encompass's both regional DC's, allowing both DC's to advertise their specific paths as well as provide a backup path for their partner DC.

    to date this has worked excellently, we've maintained 100% up time since this was deployed despite several potentially impactful outages at the ISP, CPE or facility level.

  2. We took a logically similar approach within our dual DCs. However, we manage our ONS devices for the fibre links between the DCs. So we chose to provision additional wavelengths to be dedicated for the Internet facing switches.

    Although more difficult for us to initially configure, we felt there was less risk of changes impacting the communication between the DCs because the ONS configurations are fairly static. Many more people manage our switches, and make changes much more regularly. In my company, I could more easily see someone making mistakenly changing the VRF configuration without knowing what they did.

Add comment