While running the Using VXLAN And EVPN To Build Active-Active Data Centers workshop in early December 2019 I got the usual set of questions about using BGP as the underlay routing protocol in EVPN fabrics, and the various convoluted designs like IBGP-over-EBGP or EBGP-between-loopbacks over directly-connected-EBGP that some vendors love so much.
I got a question along the same lines from one of the readers of my latest EPVN rant who described how convoluted it is to implement the design he’d like to use with the gear he has (I won’t name any vendor because hazardous chemical substances get mentioned when I do).
Aldrin wrote a well-thought-out comment to my EVPN Dilemma blog post explaining why he thinks it makes sense to use Juniper’s IBGP (EVPN) over EBGP (underlay) design. The only problem I have is that I forcefully disagree with many of his assumptions.
He started with an in-depth explanation of why EBGP over directly-connected interfaces makes little sense:
Another EVPN reader question, this time focusing on auto-RD functionality and how it works with duplicate MAC addresses:
If set to Auto, RD generated will be different for the same VNI across the EVPN switches. If the same route (MAC and/or IP) is present under different leaves of the same L2VNI, since the RD is different there is no best path selection and both will be considered. It’s a misconfiguration and shouldn’t be allowed. How will the BGP deal with this?
Got this interesting question from one of my readers:
BGP EVPN message carries both VNI and RT. In importing the route, is it enough either to have VNI ID or RT to import to the respective VRF?. When importing routes in a VRF, which is considered first, RT or the VNI ID?
A bit of terminology first (which you’d be very familiar with if you ever had to study how MPLS/VPN works):
Got an interesting set of questions from a networking engineer who got stuck with the infamous “let’s push the **** down the stack” challenge:
So I am a rather green network engineer trying to solve the typical layer two stretch problem.
I could start the usual “friends don’t let friends stretch layer-2” or “your business doesn’t really need that” windmill fight, but let’s focus on how the vendors are trying to sell him the “perfect” solution:
I’m running two workshops in Zurich in the next 10 days:
- Comparing VMware NSX and Cisco ACI (and how EVPN and VXLAN fit into the big picture) on Thursday, November 28th;
- Explaining how you could use VXLAN with EVPN to build infrastructure for active-active data centers on Tuesday, December 3rd.
I published the slide deck for the NSX versus ACI workshop a few days ago (and you can already download it if you have a paid ipSpace.net subscription) and it’s full of new goodness like ACI vPod, multi-pod ACI, multi-site ACI, ACI-on-AWS, and multi-site NSX-V and NSX-T.
Dinesh Dutt added another awesome chapter to the EVPN saga last week explaining how (and why) you could run VXLAN encapsulation with EVPN control plane on Linux hosts (TL&DR: think twice before doing it).
In the last part of current Azure Networking series I covered external VNet connectivity, including VNet peering, Internet access, Virtual Network Gateways, VPN connections, and ExpressRoute. The story continues on February 6th 2020 with Azure automation.
You’ll need Standard ipSpace.net Subscription to access both webinars.
A while ago I had an interesting discussion with someone running VMware NSX on top of VXLAN+EVPN fabric - a pretty common scenario considering:
- NSX’s insistence on having all VXLAN uplink from the same server in the same subnet;
- Data center switching vendors being on a lemming-like run praising EVPN+VXLAN;
- Non-FANG environments being somewhat reluctant to connect a server to a single switch.
His fabric was running well… apart from the weird times when someone started tons of new VMs.
In last week’s continuation of EVPN never-ending story Lukas Krattiger described how you could use EVPN constructs (VNIs, VRFs) to implement service insertion, and how you could combine then with policy-based routing.
TL&DW: It’s bridging and routing ;)
You’ll need Standard ipSpace Subscription to access the videos.
Last year I got into somewhat-heated discussion with a few engineers who followed the advice to run IBGP EVPN address family on top of an EBGP underlay.
My main argument was simple: this is not how BGP was designed and how it’s commonly used, and twisting it this way requires schizophrenic BGP routing process which introduces unnecessary complexity (even though it looks simple in Junos configuration) and might confuse people who have to run the network after the brilliant designer is gone.
Got this remark from a reader after he read the VXLAN and Q-in-Q blog post:
Another area where there is a feature gap with EVPN VXLAN is Private VLANs with VXLAN. They’re not supported on either Nexus or Juniper switches.
I have one word on using private VLANs in 2019: Don’t. They are messy and hard to maintain (not to mention it gets really interesting when you’re combining virtual and physical switches).
One of my subscribers sent me a question along these lines (heavily abridged):
My customer is running a colocation business, and has to provide L2 connectivity between racks, sometimes even across multiple data centers. They were using Q-in-Q to deliver that in a traditional fabric, and would like to replace that with multi-site EVPN fabric with ~100 ToR switches in each data center. However, Cisco doesn’t support Q-in-Q with multi-site EVPN. Any ideas?
As Lukas Krattiger explained in his part of Multi-Site Leaf-and-Spine Fabrics section of Leaf-and-Spine Fabric Architectures webinar, multi-site EVPN (VXLAN-to-VXLAN bridging) is hard. Don’t expect miracles like Q-in-Q over VNI any time soon ;)
Christoph Jaggi asked me a few questions about using VXLAN with EVPN to build data center fabrics and data center interconnects (including active/active data centers). The German version was published on Inside-IT, here’s the English version.
He started with an obvious one:
What is an active-active data center and why would I want to use an active-active data center?
Numerous organizations have multiple data centers for load sharing or disaster recovery purposes. They could use one of their data centers and have the other(s) as warm or cold standby (active/backup setup) or use all data centers at the same time (active/active).
Some (anti)patterns of network industry are way too predictable: every time there’s a new technology marketers start promoting it as the solution for every problem ever imagined. VXLAN was quickly touted as the solution for long-distance vMotion, and now everyone is telling you how to use VXLAN with EVPN to stretch VLANs across multiple data centers.
Does that make sense? It might… based on your requirements and features available on the devices you use to implement the VXLAN/EVPN fabric. We’ll cover the details in a day-long workshop in Zurich (Switzerland) on December 5th. There are still a few places left, register here.
One of my readers sent me a series of questions regarding a new cloud deployment where the cloud implementers want to run VXLAN and EVPN on the hypervisor hosts:
I am currently working on a leaf-and-spine VXLAN+ EVPN PoC. At the same time, the systems team in my company is working on building a Cloudstack platform and are insisting on using VXLAN on the compute node even to the point of using BGP for inter-VXLAN traffic on the nodes.
Using VXLAN (or GRE) encap/decap on the hypervisor hosts is nothing new. That’s how NSX and many OpenStack implementations work.