Terastream Part 2: Lightweight 4over6 and Network Function Virtualization (NFV)
In the first Terastream blog post I mentioned Deutsche Telekom decided to use an IPv6-only access network. Does that mean they decided to go down the T-Mobile route and deployed NAT64 + 464XLAT? That combo wouldn’t work well for them, and they couldn’t use MAP-E due to lack of IP address space, so they deployed yet another translation mechanism – Lightweight 4over6.
CPE Challenge
Most IPv6 residential deployments get stuck due to CPE challenges – old CPEs don’t support IPv6, it was hard to get IPv6-capable CPEs a few years ago, and it was impossible to get CPEs that supported the encapsulation/translation mechanism you wanted to use.
Deutsche Telekom found a great solution to the whole CPE conundrum: they have their own CPEs, but (if I understood the PLNOG presentation correctly) they also built OpenWRT distribution that includes lw4over6, and plan to open-source it.
AFTR Challenge
Every single ISP-scale translation or encapsulation solution requires large head-end boxes (Address Family Translation Router or AFTR in DS-Lite lingo). Deutsche Telekom decided to replace those boxes with a scale-out architecture with limited shared state (that’s why they use lw4over6 and not DS-Lite). They run numerous lwAFTR instances in the data center adjacent to a core router.
Scale-out mechanisms are extremely simple: the lw4over6 traffic arriving from the CPE routers and the return IPv4 traffic are spread across the available lwAFTR instances using the traditional 5-tuple load balancing mechanism.
Even though lwAFTR instances keep per-CPE state (mapping between IPv4 port range(s) and IPv6 CPE address), they don’t keep per-session state (NAT44 is done by the CPE). The return traffic (IPv4 to CPE) thus doesn’t have to go through the same lwAFTR instance as the outbound traffic.
Summary
Instead of investing heavily in unproven emerging technologies, Deutsche Telekom engineers designed an extremely simple and scalable access network offering IPv6 and IPv4 access. The scale-out virtual appliance architecture adapts automatically to changing user demands, minimizes the initial investment, and grows in sync with the growing user base … and they managed to do all of that with traditional technologies and standard products.
More information
- Ian Farrer’s PLNOG presentation
- Peter Lothberg’s Terastream IPv6 Details from RIPE67 (video)
The MAP-E offers a IPv4 address sharing solution for every range of IPv4 address space possible.
http://tools.ietf.org/html/draft-ietf-softwire-map-08
The difference between MAP-E and lw46 are:
- lw46 requires the operator to maintain per user 1:1 state/binding between the IPv4 address and port-set + IPv6 address. This will be done using DHCP lease state tracking, and is thus equivalent to dynamic per subscriber subscriber state on the AFTR. The mechanism will use snooping DHCPv4 running over DHCPv6, using new extensions to both protocols.
- MAP allows a range of Border-Relay configuration options, ranging from per subscriber domains, equivalent to Lw46, (i.e. One IPv4+port maps to a given IPv6 subscriber) to multi-subscriber domains (i.e. An IPv4 subnet is shared and mapped to multiple IPv6 subscribers). The latter is can be thought of IPv4 route aggregation and is a useful scalability characteristic.
In terms of provisioning MAP-E will be relying only on DHCPv6 ((http://tools.ietf.org/html/draft-ietf-softwire-map-dhcp-06), without resorting to DHCPv4 or dhcp snooping. The CPE may also be configured using TR-69.
RIPE Presentation can be found on:
https://ripe67.ripe.net/presentations/136-ripe_20_dollar_cgn.pdf
Open Source/OpenWRT on:
https://github.com/cernet/MAP
https://github.com/ayourtch/openwrt-map
Supposedly that was not the case within DT (or are you implying they implemented lw46 because they wanted to reinvent the wheel?).