L2 DCI with MLAG over VPLS transport?
One of the answers I got to my “How would you use VPLS transport in L2 DCI” question was also “Can’t you just order two VPLS services, use them as P2P links and bundle the two links into a multi-chassis link aggregation group (MLAG)?” like this:
Unfortunately, the VPLS service is never totally transparent. While you might get STP running across VPLS (but probably only if you ask), I would be extremely surprised if the CE-switches could exchange LACP/PAgP packets; these packets would be usually intercepted by the first switch in the carrier’s network.
Still, you can configure static port channel on the CE-switches and get MLAG across VPLS to work ... assuming, of course, that you’re willing to trust the Service Provider to bring down the PE-CE interface if something breaks anywhere within the VPLS cloud (otherwise your static port channel will turn into a nice partial black hole).
You could also try to run your own end-to-end tests. This is where my guesswork starts; I would appreciate comments from experts more familiar with switching platforms.
UDLD is probably out of question (yet again, one would expect UDLD packets to be received by the first provider switch), but Ethernet OAM might fit the bill. My question would thus be: can we run end-to-end Ethernet OAM on an interface that is part of a static port channel? Has anyone got that working?
More information
Numerous L2 and L3 DCI scenarios are described in the Data Center Interconnect webinar. If you’d like to know more about the benefits and drawbacks of various VPN solutions (including pseudowires, VPLS, MPLS/VPN, IPsec-based VPNs and DMVPN), check out the Choose the Optimal VPN Service webinar.
Both webinars are also available as part of the yearly subscription package.
If the reference is to my suggestion to use two VPLS instances, then I didn't have in mind running LAG over them. :) I thought of rather using them as two L3 links and doing whatever you normally would (like run your own MPLS on top of it). :)
I usually don't recommend solutions which may or may not work; hence my suggestion to use either PBB or more radically OM-5130 (which runs GFP over Ethernet/IP), as these solutions don't have dependence on service provider's capabilities to pass L2 control protocols.
Speaking of which, I did come across a few cases when an SP doesn't support Ethernet OAM, and on top of than will block your OAM frames, too.
BTW, having two VPLS services from the same SP instead of one might slightly reduce redundancy (you can't survive two worst-case access-layer failures). Having two VPLS services from two SPs is obviously a totally different story.
Last but definitely not least, obviously it's time to look into OM-5130. PBB does not solve the problem, it just introduces another layer of addressing (which does solve some other problems, but not this one).
Care to elaborate? Depending on how big and fibre-rich your provider is (and how deep your pockets are), you could potentially get your four access links going to four different POPs, with no common elements whatsoever. With smaller providers, of course, there's risk that you'll land up sharing at least some infrastructure.
> PBB does not solve the problem
I guess in my definition of "solve" it does not - it is vendor-dependent. Some vendors allow creation of MAC-in-MAC p2p (UNI to UNI) tunnels, which don't implement any L2 processing on incoming frames (just encapsulate them and ship them over, no matter what they were - control or data). But yes, it is not universal so doesn't count. :)
> it's time to look into OM-5130
If you did look at my presentation from CEW APAC '10 (I posted link in one of my responses some time ago), there was a mention of work being done at ADVA. Last year I was working with their guys on feasibility of including MPLS-based tunneling into an NTU-sized device, and there was some progress. This reminds me to get back in touch with them to see what's happening, as this discussion is a clear use case. ;)
One thing I noticed was setting EXP via service policy would not affect UDLD,STP,CDP packets. They would be set to EXP 7 where as the rest of the traffic was EXP 5 per the service policy.
To get decent reaction to one side of the one of the links going down, you need to use UDLD. Routing protocol would be preferred.
#2: Interesting. Which vendor(s) would that be? You know I'm too Cisco-focused.
#2: I'm reasonably sure Alcatel's ESS/SR can do it; and if I'm not mistaken some of Nortel's Ethernet gear could do it, too. Pretty sure there would be other implementations out there - mainly where the equipment wasn't designed as a switch from the beginning (like for example 7600/6500/Nexus) and implements bridging as a "value-add" function.
1) port to connect to a VPLS
2) exit port for QinQ
3) your "edge" port (that you'd normally connect to a VPLS).
Port 1 is connect to an SP's demarc;
Port 2 is connected to Port 3.
Port 1 carries only one VLAN - one you use on Port 2 as QinQ VLAN ID
Port 2 is configured as QinQ with L2PT p2p for PaGP/LACP
Replicate for the second SP VPLS and at the other DC.
Configure 2 x Port 3 as a member of LAG.
Pray it all works as expected! :)