Brad sent me an interesting DCI encryption question a while ago. Our discussion started with:

We have a pair of 10GbE links between our data centers. We talked to a hardware encryption vendor who told us our L3 EIGRP DCI could not be used and we would have to convert it to a pure Layer 2 link. This doesn't make sense to me as our hand-off into the carrier network is 10GbE; couldn't we just insert the Ethernet encryptor as a "transparent" device connected to our routed port ?

The whole thing obviously started as a layering confusion. Brad is routing traffic between his data centers (the long-distance vMotion demon hasn’t visited his server admins yet), so he’s talking about L3 DCI.

The encryptor vendor has a different perspective and sent him the following requirements:

1. MAC address MUST be preserved.
2. The network between encryptors cannot modify the Ethernet MAC addresses.
3. Transmission order MUST be preserved:
• QOS MUST occur outside of encryptors, not between encryptors. QOS may reorder frames.
• L2 MPLS VPN - the MPLS control word MUST be enabled to guarantee transmission order.
• L2 payload SHALL NOT be looked into by network between encryptors.

Their hardware is clearly using a proprietary encryption technology that looks like bump-in-the-wire at layer-2, so they can only work over L2 VPN offered by a Service Provider (VPLS or pseudowire). Fortunately, Brad is actually buying a L2 VPN (over which he runs L3 with EIGRP), so everything worked out just fine.

Lessons learned

1. When you buy standalone encryption devices, check whether they support IPSec or not.
2. If the encryption device does not support IPSec, it might work as a layer-3 device (router) or as a layer-2 device (bump-in-the-wire).
3. In both cases, using MPLS/VPN services from the service provider could be questionable, as you need to run PE-CE routing protocol across the encryption device.
4. It’s easiest to combine external encryptors with layer-2 VPN services (VPLS, pseudowire) or dark fiber, regardless of whether you run L2 or L3 transport across the link.
5. Even when the encryption vendor claims its device is a bump-in-the-wire, check whether it supports point-to-point or any-to-any encrypted sessions. If it’s a point-to-point device, it’s best used over a pseudowire.