Encrypting P-to-P-router traffic

Rob sent me a really good question:

I have an enterprise MPLS network. Two P routers are connected via carrier point-to-point Gigabit Ethernet and I would like to encrypt the MPLS traffic traversing the GE link. The PE-routers don't have hardware crypto accelerators, so I would like to keep the MPLS within the buildings running in cleartext and only encrypt the inter-site (P-to-P) MPLS traffic.

The only solution I could imagine would nicely fit the motto of one of our engineers: »Any time you have a problem, use more GRE tunnels« (if you have a better solution, please post it in the comments).

As far as I understand Cisco's IPSEC implementation, IOS can encrypt only IP traffic. If you want to encrypt MPLS frames forwarded by a P-router, you have to convert them back to IP ... and the only way to convert MPLS frames back into IP without losing the whole label stack which is vital to proper operation of MPLS VPN is to encapsulate them in a GRE tunnel and encrypting the MPLS-over-GRE traffic.

The "obvious" problem of this setup (apart from performance hit) is the MTU issue: the MTU on the GE link has to be high enough to support all the extra overhead on top of the 1500-byte payload. If that's not the case, you have to lower the MTU on the tunnel interface to ensure the GRE packets are not fragmented - that would kill the receiving router.

There are also "a few" intricacies of using MPLS-over-GRE/IPSec when the P-router is a 6500 or 7600. Don't ask me what they are, I'm a conceptual person and try to stay away from the specific hardware implementations, but our engineers built several large networks using this architecture and got a lot of hands-on experience. If you feel you could benefit from their expertise, get in touch with our Professional Services team.

11 comments:

  1. Basic ethernet encryption appliance will do the trick:

    http://www.cipheroptics.com/pdf/datasheet-cep1000.pdf

    The CipherEngine Enforcement Point (CEP) is a flexible encryption appliance that provides Ethernet frame encryption for Layer 2 Ethernet networks, IP packet encryption for Layer 3 networks and Layer 4 data payload encryption for MPLS networks. The CEP1000 offers full-duplex line rate encryption at 1.9Gbps using the AES encryption algorithm.

    ReplyDelete
  2. PRODUCT SNAPSHOT
    • 1.9Gbps full duplex line rate AES encryption
    • Layer 2 Ethernet frame, Layer 3 IP packet and Layer 4 payload protection
    • Preserves VLAN and MPLS tags • Create secure network groups

    ReplyDelete
  3. I've always said it is a shame this one didn't get anywhere:
    http://www.watersprings.org/pub/id/draft-raggarwa-mpls-ipsec-00.txt

    ReplyDelete
  4. Are there only 2 P with one point-to-point link??
    You can use 2 devices to do crypto (find the ones that can handle your bandwidth requirement) between your P and do xconnect of L2tpv3 over IPSec. It might sound like complex but it works. Like any tunnel solution MTU comes in the picture but with GigEth, increasing that on the P-t-P link should not be a problem. The advantage of such a solution is to keep the P configuration standard and seen as directly connected (like the CEP option mentioned by Andrew)

    Other alternative that might be worth exploring (no idea if it can match your topology):

    GET VPN: http://www.cisco.com/en/US/products/ps7180/index.html

    Andrew's proposal is interesting as well !

    ReplyDelete
  5. Yes, I like MPLS over GRE. However, we can save on 24 bytes of GRE by using MPLS over Cisco Virtual Tunnel Interface (VTI) with IPSec if there is only IP traffic.

    ReplyDelete
  6. How bout GET VPN. Now that is something that can save u some overhead. 8-)

    ReplyDelete
  7. since a long time we've been using MPLS over VTI for ALL p2p links. nowadays it is 6VPE over VTIs. Yes, we must reduce MTU on all interfaces - the same size for all,
    and it's working...

    Pls could you explain your expectation "you have to lower the MTU on the tunnel interface to ensure the GRE packets are not fragmented - that would kill the receiving router"...

    THX

    ReplyDelete
  8. If I'm not mistaken, fragmented packets are not CEF-switched and that may have an impact on the router's performance.

    ReplyDelete
  9. I can't see why this would be any different from what Cisco IOS already has (MPLS-over-GRE-over-IPSec).

    ReplyDelete
  10. these devices are doing line speed encryption in transport or tunnel mode at layer 2 and therefore allow encryption of MPLS traffic:

    http://www.atmedia.de/english/pdfs/ethernet_encryptor_flyer_en.pdf
    http://www.secunet.com/fileadmin/user_upload/Download/Printmaterial/englisch/sn_SINA_L2_FS_E.pdf

    ReplyDelete
  11. Senetas CN6100 encrypters are nice
    Thales Data Cryptor 10G encrypters are ok...

    they have some interesting features to encrypt the payload based on Ethertype == 0x8847 or 0x8848, leaving labels up to a 4 or 5 deep label stack in the clear and then encrypting the payload, while leaving IPv4 and Ipv4 in the clear... about $60k - $100k for a pair of them, depending on your discount negotiations, as of 2013.

    Why Juniper and Cisco don't build encryption of the MPLS labeled packets directly into their MPLS routers in 2013 is beyond me... Embed an FPGA directly in and implement AES, or elliptical curve crypto in the MPLS router chassis and be done.. put in knobs to leave labels in the clear and encrypt the payload.. argh!

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.