Follow-up: P-to-P router encryption

The “P-to-P router encryption” post has generated numerous comments. One of the readers suggested using dedicated Ethernet encryption devices, which is probably the best option if you’ve realized you need encryption in the network acquisition phase when there’s still some budget left (too bad the vendor recommended in the comments does not want to admit how expensive the boxes are).

However, assuming you have high-speed IPSec encryption modules and you have to implement P-to-P encryption in existing network, the only option left to you is GRE tunnel. Here’s why:

IP and MPLS are distinct protocols at the boundary between layer-2 and layer-3 (let’s forget the question whether MPLS is a layer-3 or layer-2.5 protocol at the moment). If you want to transport both of them across a shared connection, you have to include the layer-2 protocol ID (L2-PID) in the payload to ensure the receiving router is able to distinguish between incoming IP packets (which need FIB lookup) and MPLS packets (which need LFIB lookup).

As described in the previous post, IPSec encryption works only on IP packets. If you want to transport multiprotocol data (IP and non-IP traffic) across an IPSec-encrypted session, you need to encapsulate them into IP datagrams using an encapsulation method that includes L2-PID. The only such method supported in Cisco IOS is GRE.

One of the readers suggested using Virtual Tunnel Interface (VTI). The VTI is just another conceptual implementation of IPSec (using point-to-point links instead of crypto maps). It does not introduce non-IPSec encapsulation headers and thus cannot transport L2-PID and multiprotocol data. Although you can configure mpls ip on the VTI interface, it doesn’t work.

GET VPN is completely unusable for our task; it’s a scalable way of key distribution and crypto map creation. It offers no encapsulation mechanism and works only for IP traffic.

Last but not least, someone started walking down the Rube Goldberg path and proposed adding extra boxes between the P-routers and using MPSL-over-GE-over-L2TPv3-over-IPSec-over-GE-(over-VPLS). The extra devices (your Cisco AM would love this option) and different encapsulation method (L2TPv3 instead of GRE) add unnecessary complexity … unless, of course, you cannot implement hardware encryption on your P-router, in which case it’s comparable to the “dedicated Ethernet encryption devices” proposal.

8 comments:

  1. Hi Ivan,

    Interesting enough, for quite some long time there exist an RFC draft for L3 VPN PE-PE encapsulation using IPsec :)

    http://tools.ietf.org/html/draft-ietf-l3vpn-ipsec-2547-05

    However, I dont know any solution that really implements this, even though Eric Rosen is of Cisco :)

    ReplyDelete
  2. !?!?
    "...Although you can configure mpls ip on the VTI interface, it doesn’t work..."
    !?!? mistypes !?!?

    as I already wrote our 6VPE (ISIS as IGP) over VTI is perfectly working
    for ALL(!) links. BTW on link you can see only IPSec.

    ReplyDelete
  3. Yeah, I was confused by your previous comment, as I did extensive tests and it did NOT work. BTW, an interface is a VTI interface only when you have configured "tunnel mode ipsec" and "tunnel protection ipsec", otherwise it's a GRE interface with IPSec.

    ReplyDelete
  4. ! extract
    !
    interface Tunnel1109140
    description XXXXX
    bandwidth 34000
    ip address x.x.x.x 255.255.255.254
    ip mtu 1408
    ip router isis BACKBONE-2
    ...
    mpls label protocol ldp
    mpls ip
    isis circuit-type level-2-only
    isis metric 34 level-2
    ...
    tunnel source Serial4/0
    tunnel destination y.y.y.y
    tunnel key 1101140
    tunnel path-mtu-discovery
    tunnel path-mtu-discovery age-timer 13
    tunnel protection ipsec profile TUNNEL
    max-reserved-bandwidth 100
    service-policy input MPLS-CORE-INPUT
    ...
    !
    ! BTW
    the PHY-link is Ser 4/0 (E3)...

    ReplyDelete
  5. Yeah, what you have is an IPSec-encrypted GRE tunnel. Do "show interface tunnel X" and look for the "Tunnel protocol/Transport" line. See this document for a sample VTI configuration:

    http://www.cisco.com/en/US/technologies/tk583/tk372/technologies_white_paper0900aecd8029d629_ps6635_Products_White_Paper.html

    ReplyDelete
  6. We've had this in place for several years:

    2 7606's with dual ups, crypto engines and some 6748-GE-TX line cards, one in each city separated by 1000 miles, connected to each other via AT&T gigaman links (600 Mbps Ethernet transported by the SP in OC-12 channels). There are 2 TE tunnels which use FRR, then there are GRE tunnels (one per VRF) which are encrypted and transported in the TE tunnels. Most but not all of the GRE tunnels are encrypted, which was done out of concern for overrunning the crypto engine with traffic that didn't need to be encrypted. Works like a charm! and we have seen it running at full port speed on each of 2 links, done a failure test and the TE tunnel with affinity to the failed link reroutes without missing a beat. Traffic was then throttled back (retransmits and TCP window size changes) due to congestion but sessions stayed up and otherwise kept functioning without any issues.

    ReplyDelete
  7. You can also use a NON-GRE VTI interface on the PEs to encrypt IPSEC packets and then label switch them to the other PE.

    http://blog.ippacket.info/2010/03/mpls-l3-vpn-and-encryption-on-the-pe-router/

    -Rakesh.

    ReplyDelete
  8. Ivan Pepelnjak19 June, 2010 10:50

    Truly interesting hack ... but utterly non-scalable. Not to mention the potential for recursive routing within a VRF. It would also be interesting to see how this performs on high-end (hardware switching) boxes.

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.