Why Exactly Would You Want a Nexus 7000 in There?

Network designers (and smart consulting and system integration companies) often use ExpertExpress to get a second opinion on a design someone put together using technologies they’re not thoroughly familiar with. Not surprisingly, some of those third-party designs aren’t exactly optimal.

A while ago I was asked to review a data center “design” proposed to my customer by a system integrator. It had a pair of Nexus 5500 switches connecting servers and storage to a single Nexus 7000, which was then connected to WAN edge routers.

I tried to understand what the role of the Nexus 7000 was, as it looked more like a really expensive bump-in-the-wire in that particular design (not to mention it also acted as a nice single-blob-of-failure). The only potential use cases we could see for that box were:

  • Layer-3 forwarding, which could be performed on either Nexus 5500 switches or WAN edge routers;
  • OTV for potential data center extension, and the customer didn’t have the second data center or immediate plans to deploy one.

Please note I’m not picking on the Nexus 7000 or its features. It’s just that it was not needed in that particular design. On the other hand, don’t try to persuade me how redundant supervisors in a single box remove a single point of failure.

At that time OTV was already available on ASR1K routers, and the customer already had plans to replace existing WAN edge routers with ASR1Ks, so the OTV argument was gone. We analyzed the potential drawbacks of running layer-3 forwarding on either Nexus 5500s (performance limitations of layer-3 daughterboard and lack of ISSU), but quickly realized that:

  • The 256160 Gbps of layer-3 throughput offered by Nexus 5500 was more than enough;
  • ISSU was a non-issue – the customer could easily get the job done with a single Nexus 5500; temporarily losing the other box wasn’t such a big deal.

End result: the customer saved several hundred thousand dollars with a single one-hour consulting session.

I know it was an extremely low-hanging fruit, but I simply couldn’t resist writing about it.

Recent posts in the same categories

20 comments:

  1. Unknown 24 April 2014 08:52
    One other issue on the Nexus 5500 L3 Platform could be very very small TCAM for things like RACL etc.
    Replies
    1. Paul Stewart 24 April 2014 13:44
      I agree. The TCAM can be a huge issue on the 5500 if you do much with ACL's. This is especially true if you use port ranges.
  2. Unknown 24 April 2014 16:01
    The 5600 solves a lot of the layer 3 problems the 5500 has. It doesn't require a daughter card and has line-rate layer 3. The 5600 is just a re-branded 6001.
    Replies
    1. Greg Ferro 24 April 2014 18:01
      I didn't know that. Thanks!
    2. Unknown 25 April 2014 08:28
      The 6001 are even worse around RACL because of the way the chips are layed out. If you use SVI the switch will have to install the RACL in all chips in the switch
  3. Anonymous 24 April 2014 16:03
    The designer might think creating two different VDC one for data center with FabricPath one for WAN aggregation or services.
    Replies
    1. Greg Ferro 24 April 2014 18:01
      This is a good point, but at the cost of NX7K (and it's maintenance) for VDC it's cheaper to buy 2x NX5K and have a separate physical switches. And more reliable.
  4. Anonymous 24 April 2014 20:50
    We built something just like this for our dedicated backup network. We use the typical 7k/5k/2k in the server environments, but we wanted a dedicated backup network, shared by Prod and non-Prod, operates 24/7, no impact to prod bandwidth. We basically decided on an out-of-band switch stack with dedicated cabling. The extremely simplified layer-3 requirements and separate routing table completely justified the decision to use a pair of Distribution 5ks with L3 modules and a LOT of Access 5ks. We realized a huge savings over the 7k on a cost-per-10gb-port basis. It's less oversubscribed too.

    Incidentally, when we inquired about future Layer-3 for 5k platforms, we learned that 5k and 7k were different internal business units at Cisco and there was quite a bit of contention as the platforms overlapped. We were thinking that if they just add a few more features, we'd probably use the 5ks far more frequently for L3-switching .

    CWB
  5. Oliver 25 April 2014 05:16
    In general...I'm not sure that "one" of anything that relates to production makes a whole lot of sense in an actual data center. If you can't afford to buy two of them, you should probably look at another platform/design.

    Similar to the last anonymous poster, we did use a single Nexus 7000 per site to build a separate backup network at one point. And then our app teams proceeded to use the non-redundant "backup" network to access NFS shares for databases used by mission critical applications...
  6. Unknown 25 April 2014 07:01
    One could've readily replace the Nexus 7000 with any potential core switch and one of the main arguments is still equally valid if your core is a SPoF.
  7. Shane Hale 25 April 2014 15:39
    Nice Write up thanks!!
  8. Anonymous 25 April 2014 18:25
    Its already been said but if you don't need native FiberChannel, try out the Nexus 6001 or wait for the 5600 series to be released for sale. Or better yet, check out the bundle pricing on two Nexus 9396, they are almost giving them away to get people in line for ACI (can be installed in stand-a-lone mode if not ready for ACI).
  9. Anonymous 25 April 2014 19:58
    Arista would have been far better and less expensive.
  10. Kevin Breit 26 April 2014 02:30
    My understanding is the Nexus 5500 has 160Gbps of layer 3 throughput. Where is the 256Gbps coming from? Am I incorrect?
    Replies
    1. Ivan Pepelnjak 26 April 2014 09:22
      My mistake (it's been a while ...). Fixed. Thanks for pointing it out.
  11. Anonymous 26 April 2014 12:49
    That's also another use case where using something else than Cisco would have been better (linerate L2/L3) and cheaper, in such design.

    As for the SPOF, I agree, a single box is a SPOF. At a different degree depending on hardware and software redundancy, but still a SPOF. A single Control Plane with 2 physical devices is also a SPOF.
  12. Eric 26 April 2014 19:20
    I think you are correct Kevin. The 6001 os on the other hand has L3 forwarding capabilities of 1.28Tbps IIRC.
  13. Anas 01 May 2014 05:21
    I agree, no need for a Nexus 7k in this case. For me though I prefer not to route on the Nexus 5500 in general especially when I'm using Nexus 2200 as ToR because the Nexus 5500 supports only 16 FEX's with layer 3 enabled (compared to 24 in layer 2). If you do the math losing 8 FEX's per Nexus 5500 could mean losing a lot of server ports.
  14. Erik 08 May 2014 17:15
    The only time a 7k makes it into the design, when the boss already bought it and I am making the best of using a star destroyer to kill an ewok
  15. Martin0641 18 February 2016 07:00
    The only thing I can think of is advanced layer 3 capabilities like QoS and being about the set the MSS in case of GRE tunnels. But you could use a much cheaper ASR for that.
Add comment
Home
Sidebar