MPLS/VPN Common Services Design

The Common Services MPLS/VPN topology is the topology in which multiple customers access the same common servers without being able to access each other’s networks. You can implement this requirement with judicious use of inter-VRF NAT or with controlled route leaking between customers’ and common services VRFs (assuming the customers don’t use overlapping address space).

A simple implementation of Common Services topology adds customer route targets to the common services VRF (Jeremy Stretch from Packetlife.net wrote an excellent in-depth description of this setup). Modifying common services VRF whenever a new customer is added obviously does not scale; a truly scalable solution uses asymmetrical import/export route targets as explained in the following video:


More information

7 comments:

  1. Interesting enough I have just spoken about a similar design with a global MPLS provider and they specifically stated they will not put any special import/export (and certainly not import/export route-maps) on their PE routers. Their argumentation is that they can only provide standard solution which is templated on their PE routers, and adding couples "special" lines under my VRFs to 1000+ PE routers creates an unmanagable solution for them.....

    ReplyDelete
  2. Ivan Pepelnjak18 May, 2011 16:17

    ... and then there are those that wonder why anyone ever contemplates anything as stupid as running private MPLS/VPN cloud over IP over public MPLS/VPN cloud. :-E Because the SP's can't deliver :'(

    ReplyDelete
  3. I work with an enterprise customer that has implemented a self-managed RFC 4364BGP/MPLS IP VPNs as an overlay an exiting SP's cloud even though the SP offers the same service. This was to empower the enterprise's engineers with control over routing and VRF isolation.

    ReplyDelete
  4. This is an overly complex and unsupportable approach to shared services. Having to touch 650+ PE routers and thousands of VRFs to create a shared services VPN is unacceptable. The correct approach is to touch only the "services" vrf, and import/export to each RT that you wish to insert the services into. Furthermore, a route-map MUST be applied to the services VPN blocking RFC1918 space, as well as the default route and any other overlapping prefixes. For examples where the shared services VPN is for management and monitoring of CPE devices, there must be globally unique addressing between all VPNs for the endpoints that need to communicate with the shared services VPN. If we learn the 10.1.1.0/24 prefix from 20 customers, hosts in those networks will be unable to use shared services.

    ReplyDelete
  5. Ivan Pepelnjak19 May, 2011 09:53

    Interesting point of view. As always, there is no "right" answer; it depends on whether you have several large customers (in which case adding customer RT to CS VRF makes sense) or many small ones (in which case per-CS asymmetric RT works better).

    It also depends on whether you provision common services at the same time you create customer VRFs or at a later time ... and also whether you use an automatic provisioning system or configure VRFs by hand.

    As for RFC 1918 and overlapping addresses - I clearly stated that in the very first paragraph. In most cases inter-VRF NAT is a safer option (how many MPLS VPN customers do you have that use public IP addresses in their network?)

    ReplyDelete
  6. In the past when we deploy managed endpoints, such as handsets for managed telephone service, or managed CPE routers, the phones and routers were assigned globally routable address space. This was due to the common services vrf design and its failures.

    For new customers that design is superceded for the phones by use of session border controllers with back-to-back user agent configuration. Now that SBCs are starting to support fully virtualized routing tables just like the PE's the duplicate address space is a non-issue, and the SBCs drastically improve compliance with CALEA.

    The same is true for managed services monitoring, instead of using common services vrfs and the trouble they bring, we just use vmware to spin up a probe with one leg on the customer network and one leg on the management network to handle all the administrative tasks.

    All in all, the common services VRF concept should be retired. It created far more problems than it solved, and better solutions (more secure, more scalable, more managable) are available now.

    ReplyDelete
  7. Derick Winkworth25 May, 2011 17:30

    I agree and disagree at the same time. The common services VRF as is commonly depicted is severely flawed, but you can't get around the fact that highly leveraged servers/load-balancers/etc exist in an IP cloud separate from your customers. You can't put public addressing on thousands of IP endpoints in your services environment.

    I think with some modifications the shared services model can work wonderfully. If you are in the business of pushing out IP apps/services to thousands of customers then at some point you need to accept some complexity as you are basically intersecting with thousands of networks.

    1. NVI is a no-go if you need customized NAT (can't reuse global address from one VRF to another). Virtualize your NAT router with "match-in-vrf" and you can customize NAT and inject routes as needed from one point. With our design we can even control bandwidth into the services environment on a per-customer basis on the NAT router.

    2. Route-leaking is a terrible idea. Just give each shared services environment a pair of redundant PEs, assymetric route-targets (as described in Ivan's post) and be done with it.

    3. Automation. Quit being cheap bastards and build tools for EMAC work. Hire a linux/perl geek and throw up a site built on catalyst. Utilize the tools that vendors provide you (on-box Tcl, XML-PI, JUNOScript).

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.