Data Center Interconnect (DCI) encryption

Brad sent me an interesting DCI encryption question a while ago. Our discussion started with:

We have a pair of 10GbE links between our data centers. We talked to a hardware encryption vendor who told us our L3 EIGRP DCI could not be used and we would have to convert it to a pure Layer 2 link. This doesn't make sense to me as our hand-off into the carrier network is 10GbE; couldn't we just insert the Ethernet encryptor as a "transparent" device connected to our routed port ?

The whole thing obviously started as a layering confusion. Brad is routing traffic between his data centers (the long-distance vMotion demon hasn’t visited his server admins yet), so he’s talking about L3 DCI.

The encryptor vendor has a different perspective and sent him the following requirements:

  1. MAC address MUST be preserved.
  2. The network between encryptors cannot modify the Ethernet MAC addresses.
  3. Transmission order MUST be preserved:
    • QOS MUST occur outside of encryptors, not between encryptors. QOS may reorder frames.
    • L2 MPLS VPN - the MPLS control word MUST be enabled to guarantee transmission order.
    • L2 payload SHALL NOT be looked into by network between encryptors.

Their hardware is clearly using a proprietary encryption technology that looks like bump-in-the-wire at layer-2, so they can only work over L2 VPN offered by a Service Provider (VPLS or pseudowire). Fortunately, Brad is actually buying a L2 VPN (over which he runs L3 with EIGRP), so everything worked out just fine.

Lessons learned

  1. When you buy standalone encryption devices, check whether they support IPSec or not.
  2. If the encryption device does not support IPSec, it might work as a layer-3 device (router) or as a layer-2 device (bump-in-the-wire).
  3. In both cases, using MPLS/VPN services from the service provider could be questionable, as you need to run PE-CE routing protocol across the encryption device.
  4. It’s easiest to combine external encryptors with layer-2 VPN services (VPLS, pseudowire) or dark fiber, regardless of whether you run L2 or L3 transport across the link.
  5. Even when the encryption vendor claims its device is a bump-in-the-wire, check whether it supports point-to-point or any-to-any encrypted sessions. If it’s a point-to-point device, it’s best used over a pseudowire.

More information

Choose the Optimal VPN Service webinar (register here) describes numerous VPN services, including (layer 3) MPLS/VPN, (layer 2) VLPS and pseudowires. It also explains how each services looks from the enterprise network perspective and how it’s best used.

Data Center 3.0 for Networking Engineers webinar (buy a recording or yearly subscription). covers (among tens of other topics) four DCI scenarios, including pseudowires, VPLS-based solutions, OTV and multiple user-side EoMPLS variants. The Data Center Interconnects webinar goes even deeper into the DCI topics and describes active-active data center designs.

5 comments:

  1. Which vendor is it?

    ReplyDelete
  2. Christoph Jaggi28 October, 2010 14:29

    There is a range of different ethernet encryption appliances available on the market. In terms of 10G point-to-point there are currently 3 platforms and 7 vendors. The platform developers are ATMedia (used by ATMedia, Secunet and Thales), Infoguard/Crypto (used by Infoguard) and Senetas (used by Senetas, Safenet and IDQ).

    An overview of the different offers can be found here:

    http://uebermeister.com/files/inside-it/2010_Uebersicht_Verschluessler_Ethernet_P2P.pdf

    ReplyDelete
  3. Thank you for the link. It's a fantastic summary.

    ReplyDelete
  4. Christoph Jaggi30 October, 2010 10:52

    There is also an overview of the available ethernet multipoint encryption appliances:

    http://uebermeister.com/files/inside-it/2010_Uebersicht_Verschluessler_Ethernet_Multipunkt.pdf

    In addition, there are explanations to both overviews (in German only) and a bunch of other documents (also currently in German only). All of them have been published on www.inside-it.ch.

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.