Every time I write about IPv6 multihoming issues and the need for NPT66, I get a comment or two saying “but I thought this is already part of IPv6 stack – can’t you have two or more IPv6 addresses on the same interface?” The commentators are right, you can have multiple IPv6 addresses on the same interface; the problem is: which one do you choose for outgoing sessions.
The source address selection rules are specified in RFC 3484 (Greg translated that RFC into an easy-to-consume format a while ago), but they are not very helpful as they cannot be influenced by the CPE router. Let’s look at the details.
Phase 1 – single ISP connection
We have a simple SMB network: a single CPE router connected to one ISP and a host sitting behind the router (ignore the PE-B part for the moment). CPE router asks ISPA for a delegated prefix (using IA_PD option in DHCPv6) and uses part of that prefix to address its LAN interface.
This is how you configure the CPE router if you’re using Cisco IOS:
ipv6 unicast-routing ! interface FastEthernet0/0 description Inside interface ipv6 address ISPA ::1/64 ipv6 nd router-preference High ipv6 nd ra interval 10 ! interface FastEthernet1/0 description ISP A uplink ipv6 address autoconfig default ipv6 dhcp client pd ISPA
The CPE router configuration is not complete; you would also need DHCPv6 server on the inside interface to pass DNS server IPv6 address to the clients. A complete (and tested) configuration is included in the materials you get with the Building IPv6 Service Provider Core webinar.
The IPv6 client receives RA messages sent by the CPE and creates an IPv6 address from the advertised /64 prefix on its LAN interface:
The client is now able to communicate with the IPv6 Internet. Problem solved ... until someone figures out a single upstream connection is a single point of failure and orders a second Internet service.
Phase 2 – Two ISP uplinks
The second ISP uplink is configured almost identically to the first. Since you cannot have two RA-generated default routes in Cisco IOS release 15.1M, I had to use a floating static default route and hard-code the next-hop router’s IPv6 address in it.
ipv6 unicast-routing ! interface FastEthernet0/0 description Inside interface ipv6 address ISPA ::1/64 ipv6 address ISPB ::1/64 ipv6 nd router-preference High ipv6 nd ra interval 10 ! interface FastEthernet1/0 description ISP A uplink ipv6 address autoconfig default ipv6 dhcp client pd ISPA ! interface FastEthernet1/1 description ISP B uplink ipv6 address autoconfig ipv6 dhcp client pd ISPB ! ipv6 route ::/0 FastEthernet1/1 FE80::2 30
After the CPE router receives a delegated prefix from PE-B, it adds a /64 prefix from that address range to its LAN interface and starts advertising two /64 prefixes (one from each ISP) in its RA messages. The IPv6 client creates a second IPv6 address from the second advertised prefix – it now has two IPv6 addresses on its LAN interface.
There are numerous problems associated with this setup, some of them architectural, many more due to suboptimal implementations, omissions, or strict adherence to RFCs in host and router stacks.
The easy one first: when an IPv6 client with multiple IPv6 addresses starts a new session, it chooses a source address that best matches the destination address (ULA address for ULA destination, global address for global destination ...) without any knowledge of the network topology.
Distributing Address Selection Policy using DHCPv6 draft describes a potential solution ... but it has to be implemented in both routers and hosts, and it’s implemented nowhere at this moment.
For example, when the IPv6 client in our small network connects to the outside world, it might choose a source IPv6 address assigned by the wrong ISP.
You can see the source address used by the client with the netstat -n -p tcpv6 (Windows) or netstat -n -f inet6 -p tcp (Linux/OSX) command. It seems that Windows picks the lowest IPv6 address while OSX picks the oldest IPv6 address when all interface IPv6 addresses are equivalent according to RFC 3484 rules.
The best that could happen is asymmetrical routing:
In some rare cases, the ISP actually performs RPF check and drops the packet with an unexpected source IPv6 address.
The whole situation might have been survivable were this the only problem to solve (and the lack of RPF checks on the ISP side causes people to claim that IPv6 multihoming works). Unfortunately, there are many others, for example:
- When the CPE router (Cisco IOS router running 15.1(4)M) loses an uplink, it does not stop advertising the delegated prefix it received through that uplink (implementation issue). One of the client IPv6 addresses is thus completely invalid without client being aware of it.
- If you clear the delegated prefix manually (or with an EEM applet) on the CPE router, it stops advertising the prefix in its RA messages ... but the prefix remains valid on the IPv6 hosts until it expires (architectural issue). Prefix expiration is based on its preferred lifetime, which is derived straight from DHCPv6 prefix delegation and is usually measured in weeks.
- It might be possible to reduce the preferred lifetime in the RA messages to a very low number, but the lifetime of an interface prefix based on a delegated prefix is not configurable (implementation issue).
Please don’t try to tell me that the whole thing works if you use two CPE routers. It might work once the host stacks implement RFC 3484 bis, but we’re not there yet (and I’ll describe that scenario in an upcoming blog post).
More information and tested router configurations
Do your own tests
If you want to test how your hosts behave in this scenario or try to fix my router configurations, use these configurations as a starting point:
version 15.1 service timestamps debug datetime msec service timestamps log datetime msec no service password-encryption ! hostname CPE ! ipv6 unicast-routing ipv6 cef ! interface FastEthernet0/0 description Inside interface ipv6 address ISPA ::1/64 ipv6 address ISPB ::1/64 ipv6 nd router-preference High ipv6 nd ra interval 10 ! interface FastEthernet1/0 description ISP A uplink ipv6 address autoconfig default ipv6 dhcp client pd ISPA ! interface FastEthernet1/1 description ISP B uplink ipv6 address autoconfig ipv6 dhcp client pd ISPB ! ipv6 route ::/0 FastEthernet1/1 FE80::2 30 ! line con 0 exec-timeout 0 0 privilege level 15 line vty 0 4 exec-timeout 0 0 privilege level 15 no login ! ntp logging end
hostname PE ! ipv6 unicast-routing ipv6 cef ipv6 dhcp pool ISPA prefix-delegation pool ISPA ! ipv6 dhcp pool ISPB prefix-delegation pool ISPB ! interface Loopback0 ipv6 address 2001:DB8:CAFE::1/64 ! interface FastEthernet0/0 description ISP A interface ipv6 address FE80::1 link-local ipv6 address 2001:DB8:1:FF01::1/64 ipv6 dhcp server ISPA ! interface FastEthernet0/1 description ISP B interface ipv6 address FE80::2 link-local ipv6 address 2001:DB8:7:FF01::1/64 ipv6 dhcp server ISPB ! ipv6 local pool ISPA 2001:DB8:1::/49 60 ipv6 local pool ISPB 2001:DB8:7::/49 60 ! line con 0 exec-timeout 0 0 privilege level 15 line vty 0 4 exec-timeout 0 0 privilege level 15 no login ! ntp logging end