A while ago I had an interesting consulting engagement: a multinational organization wanted to migrate off global Carrier Ethernet VPN (with routers at the edges) to MPLS/VPN.
While that sounds like the right thing to do (after all, L3 must be better than L2, right?) in that particular case they wanted to combine the provider VPN with Internet-based IPsec VPN… and doing that in parallel with MPLS/VPN tends to become an interesting exercise in “how convoluted can I make my design before I give up and migrate to BGP”.
Remember how Arista promoted VXLAN coupled with deep buffer switches as the perfect DCI solution a few years ago? Someone took Arista’s marketing too literally, ran with the idea and combined VXLAN-based DCI with traditional MLAG+STP data center fabric.
While I love that they wrote a blog post documenting their experience (if only more people would do that), it doesn’t change the fact that the design contains the worst of both worlds.
Here are just a few things that went wrong:
Every time I’m running a data center-related workshop I inevitably get pulled into stretched VLAN and stretched clusters discussion. While I always tell the attendees what the right way of doing this is, and explain the challenges of stretched VLANs from all perspectives (application, database, storage, routing, and broadcast domains) the sad truth is that sometimes there’s nothing you can do.
In those sad cases, I can give the workshop attendees only one advice: face the reality, and figure out how badly you might fail. It’s useless pretending that you won’t get into a split-brain scenario - redundant equipment just makes it less likely unless you over-complicated it in which case adding redundancy reduces availability. It’s also useless pretending you won’t be facing a forwarding loop.
I had no specific expectation when I started watching the material and I must have watched it 6 times by now.
Your webinar covered just the right level of detail to educate myself or refresh my knowledge on the technologies and relevant options for today’s market choices
The information provided is powerful and avoids useless discussions which vendors and PowerPoint pitches. Once you ask the right question it’s easy to get an idea of the vendor readiness
In the first live session we covered the easy cases: design considerations, and layer-3 interconnect with path separation (multiple routing domains). The real fun will start in the second live session on March 19th when we’ll dive into stretched VLANs and long-distance vMotion ideas.
A while ago we published a guest blog post by Christoph Jaggi explaining the high-level security challenges of most SD-WAN solutions… but what about the low-level details?
TL&DW: some of the SD-WAN boxes are as secure as $19.99 Chinese webcam you bought on eBay.
Here’s some feedback I got from a subscriber who got pulled into an SD-WAN project:
I realized (thanks to you) that it’s really important to understand the basics of how things work. It helped me for example at my work when my boss came with the idea “we’ll start selling SD-WAN and this is the customer wish list”. Looked like business-as-usual until I realized I’ve never seen so big a difference between reality, customer wishes and what was promised to customer by sales guys I never met. And the networking engineers are supposed to save the day afterwards…
How did your first SD-WAN deployment go? Please write a comment!
You need at least free ipSpace.net subscription to watch the video.
Christoph Jaggi, the author of Transport and Network Security Primer and Ethernet Encryption webinars published a high-level introductory article in Inside-IT online magazine describing security deficiencies of SD-WAN solutions based on the work he did analyzing them for a large multinational corporation.
As the topic might be interesting to a wider audience, I asked him to translate the article into English. Here it is…
Andy sent me this question:
I'm currently playing around with BGP & VXLANs and wondering: is there anything preventing from building a virtual IXP with VXLAN? This would be then a large layer 2 network - but why have nobody build this to now, or why do internet exchanges do not provide this?
There was at least one IXP that was running on top of VXLAN. I wanted to do a podcast about it with people who helped them build it in early 2015 but one of them got a gag order.
One of my readers sent me this question:
I'm in the process of researching SD-WAN solutions and have hit upon what I believe is a consistent deficiency across most of the current SD-WAN/SDx offerings. The standard "best practice" seems to be 60/180 BGP timers between the SD-WAN hub and the network core or WAN edge.
Needless to say, he wasn’t able to find BFD in these products either.
Does that matter? My reader thinks it does:
One of my friends sent me this design challenge:
Assume you’re migrating from another WAN transport technology to MPLS. The existing network has 3000 routes but the MPLS carrier is limiting you to 1000 routes. How could you solve this with MPLS?
Personally, I think MPLS is a red herring.
A better question would be “how do you reduce the number of routes transported across your WAN network” or “how do you reduce the routing interaction with your MPLS service providers” (particularly intriguing if you use more than one of them).
As always, there are several options and it’s impossible to recommend the best one:
- Readdressing is usually out of question (or at least too messy to try). It might also break numerous firewall rules and other hard-coded stuff… unless you automated everything, but then it wouldn’t be hard to readdress, would it?
- The usual answer would be to summarize the routes. The usual challenge is that you might not be able to do it (because random addressing). Furthermore, summarization is a lossy compression, and loss of forwarding information might result in black holes.
- RFC 1925 states that there’s nothing that cannot be solved with another layer of abstraction. In this case, we could use any one or more of a half-dozen overlay technologies (IPsec, GRE, VXLAN, DMVPN, LISP…), or use an overlay technology sprinkled with unicorn dust (aka SD-WAN). The beauty of CE-to-CE tunnels is that they totally eliminate the need for PE-CE routing, and (when combined with VRFs) create independent routing domains, so you can use multiple SPs without the associated hassle.
- Finally, you could go for a really exotic solution like Carriers-Carrier (using additional MPLS labels as the data-plane abstraction mechanism).
Another question from someone watching my Designing Active-Active and Disaster Recovery Data Centers webinar (you know, the one where I tell people how to avoid the world-spanning-layer-2 madness):
In the video about parallel application stacks (swimlanes) you mentioned that one of the options for using the R/W database in Datacenter A if the user traffic landed in Datacenter B in which the replica of the database is read-only was to redirect the user browser with the purpose that the follow up HTTP POST land in Datacenter A.
Here’s the diagram he’s referring to:
Michael Klose left an interesting remark on my Regional Internet Exits in Large DMVPN Deployment blog post saying…
Would BGP communities work? Each regional Internet Exit announce Default Route with a Region Community and all spokes only import default route for their specific region community.
That approach would definitely work. However, you have to decide where to move the complexity.
One of my readers was watching the Building Active-Active Data Centers webinar and sent me this question:
I'm wondering if you have additional info on how to address the ingress traffic flow issue? The egress is well explained but the ingress issue wasn't as well explained.
There’s a reason for that: there’s no good answer.