Dynamic FCoE – Sparse-Mode FCoE Strikes Again
A while ago Cisco added dynamic FCoE support to Nexus 5000 switches. It sounded interesting and I wanted to talk about it in my Data Center Fabrics update session, but I couldn’t find any documentation at that time.
In the meantime, the Configuring Dynamic FCoE Using FabricPath configuration guide appeared on Cisco’s web site and J Metz wrote a lengthly blog post explaining how it all works, triggering a severe attack of déjà vu.
I called Cisco’s traditional approach to multi-hop FCoE dense-mode FCoE – every switch in the network is a Fibre Channel Forwarder (FCF – Fibre Channel equivalent of a router) with a unique Domain ID participating in FSPF.
Tony Bourke wrote an even more structured blog post on multihop FCoE topologies. Definitely worth reading.
Dynamic FCoE is Cisco’s implementation of sparse-mode FCoE (note that my blog post describing it is four years old) – the leaf switches (Nexus 5000s) are FCFs, core switches provide layer-2 FabricPath-based transport between the leaf switches. There really is nothing new under the sun ;)
Does dynamic FCoE make sense? It depends. If you’re willing to trust the layer-2 transport fabric and treat storage as yet another application on top of Ethernet+IP, dynamic FCoE makes perfect sense, but keep in mind that you’ll have as much visibility into underlying transport fabric as with iSCSI or NFS… or overlay virtual networks.
More information
FCoE is just one of the many topics described in the storage section of my Data Center 3.0 for Networking Engineers webinar.
Maybe I'm wrong, but most of the customers we meet today are not worried about the health of their storage network. They just power it on and hope it'll work as designed. As soon as it works, they don't touch it any more except when they have to add new F_Ports or they are experiencing compatibility issues.
Multihop FCoE is lacking of monitoring tools today, but I'm pretty sure that we have exactly the same issue with FC networks. The only things that differs is you have dedicated networks for those type of traffic (FC and Dedicated links FCoE), so congestion should be quite simple to deal with.
With Dynamic FCoE, to me, the only sexy features you have are :
- no more storage licences on the spine
- high availibility of each fabric even in case of spine failure (with guaranteed isolation, cf J Metz blog post)
- lots of bandwidth between the core and edge (N*10G/40G)
- REAL mutualized infrastructures, that could be a disadvantage in some cases (lack of visibility, QoS, ...) but in most cases is really appreciated by customers (they don't understand the marketing term Unified Fabric and the fact you have to dedicate links for FCoE)
But I strongly agree with you, you won't have more visibility that you already have today... The day we will have tools to guarantee to applications that network is clearly not concerned about performance issues in FC/FCoE storage networks, we'll have done a big step I think!
Pierre-Louis
For the Dynamic FCoE configuration, does the leaf switch have to be an FCF? Can we use a generic non-FCoE aware switch as leaf to connect hosts?
Thanks.
Ben
Please let me know if my understanding below makes sense:
Assume the leaf switch has to be FCF, then will the multi-vendor network be an issue? For example, I have a network environment with 40G/100G spine and 10G leaf all from one vendor but they are not FCoE aware but support DRILL (or maybe Cisco Fabric Path). Then I want to incorporate FCoE FCF, which is from another vendor but also support DRILL or Fabric Path, into existing environment, does that mean we have to re-connect host cable for existing non-FCF switch to the FCF?
Thanks.
Ben.
It seems FCoE community latest draft comes up with something like FDF, which looks to me like a simplified FCoE aware bridging device that can uplink to FCF and downlink to hosts. Looks to me Cisco is what behind to drive FDF/FCF (they call it FCDF) standard so I guess once it comes into GA product it will add more complexities and cause more confusions and compatibility issues. Maybe I'm wrong but sounds to me FCoE community is repeating the same compatibility mistakes as they did in FC SAN.
Ben.