Whitebox Switching and Industry Pundits
Industry press, networking blogs, vendor marketing whitepapers and analyst reports are full of grandiose claims of benefits of whitebox switching and hardware disaggregation. Do you ever wonder whether these people actually practice their theories?
There’s a simple litmus test: look at the laptop they’re using. Is it a no-name x86 clone or a brand name laptop? Is it running Linux or Windows… or are they using a MacBook with OSX because it works better than any alternative (even Windows runs better on a MacBook than on a low-cost craptop). Finally, are they using Android on a low-cost ODM phone or an iPhone?
If you're using Apple products while promoting whitebox switching, it would be nice to hear your reasoning. Please leave a comment!
So my phone *is* an Android phone (I'm too much of a coward to go all the way to Chinese no-name stuff that only runs Cyanogenmod, and too much of a cheapskate to but a "truly open" Firefox-or-whatever phone).
And yes, I fully expect my next ToR switches will be called "Edge-Core" or "Quanta" rather than Arista, Cisco or Juniper...
All the best!
And in our Lab, we have installed our first 10 GE Edge-corE switch which runs ONIE and various Linux based network OS. So far, I am quite happy with what we experienced with it, though hoping that Open Network Linux will soon include a forwarding plane that is apparently in the works with the support of Broadcom and Big Switch, among others.
It's a pets and cattle argument. My phone and laptop are pets, my network should be cattle.
Thank you!
I think there are more shades of grey, but in a sense, yes. Whitebox value comes in flexibility, price and ease of automation. If you are running EIGRP on Cat6k and managing with Cisco Security Manager, whitebox probably isn't a good fit for your org.
The company needs both the technical drivers as well as the staff will to make the change.
I get a typical Steve Job's answer that the hardware and software should be owned by the same vendor/organization. I was told that the the vendor managing hardware and software do well in terms of silicon/chip failures from cosmic radiation. Apparently it seems cosmic radiation causes downtime and the Big-5 have some secret to prevent, which they claim the whitebox vendors will unable to do it.
I was even presented mathematics of how whitebox vendors are losing money and how their business models are not good.
Do you have any comparative study describing the pros and cons of using britebox/whitebox switches
I would love to see a comparative study, but doubt we'll see an unbiased one in a long long time. We could, however, use Linux and its acceptance/success in various market segments as an approximation (keeping in mind Linux development started in 1991 ;).
The cosmic radiation thing is hilariously bogus.
Several of the name-brand switch vendors have their ODM partner ship directly from the factory to the customer. In this case, the only differences between the branded version and the Open Networking version are:
1) ONIE preinstalled vs vendor OS preinstalled
2) Whether or not the vendor logo is applied.
=====
Apparently the cause of the ACE failure was due to an environmental
condition causing a 1-bit flip which the ACE detected as a parity error.
The probability of a bit flipping has to do with the level of background
radiation, which is a function of the height above sea-level, the amount
of concrete around, and many other environmental factors.
This issue cannot be fixed until Cisco either does a significant Itasca
re-design or moves the data-path to Miltons (which do not have SRAM).
This re-design is about a year away.
Currently a reboot is needed to resolve the problem, which the ACE did
on its own. The ACE failed over has expected and is configured to
maintain active sticky connections.
=====
This trend seems to happen to almost all areas of technology: once cheap readily available hardware reaches a certain level of performance/capability, hardware itself becomes less of a differentiator and the software becomes the focus instead.
Whitebox switching is starting to become a reality as the necessary hardware is on the cusp of reaching that level of capability (although as mentioned there are many areas in this ecosystem that need to develop for it to become more mainstream).
If your business is completely risk-averse, it's unlikely you'll do more than look at whitebox. We are currently heavy users of vblock. It's all off the shelf stuff and familiar to our staff... could we have built vblocks on our own? Technically, yes... realistically, no. Our business process, our organization, and our risk aversion hampered a smooth adoption of private cloud, so we went the "managed" route. However... the experience has opened up minds and imagination. We have gradually assumed more responsibility for the vBlocks and reduced our suppport dependency. Now, Whitebox/britebox is mentioned in every roadmap discussion nowadays... and that's a good thing.
My opinion, even if you aren't Facebook - with a laser-focused design goal and top notch support staff - you can still consider it. It doesn't always make sense to put Production-grade infrastructure in you non-production environments, so that is a good candidate to kick the tires on whitebox/britebox. If it works well, obviously the next step is to use it in production. The cost savings are pretty apparent, but there is absolutely some technical debt.
You assume two things falsely:
1) Whitebox hardware is == to "craptop" hardware.
2) Desktop OSs are consumed the same as infrastructure OSs.
The reality:
1) Most popular WB hardware comes off of the same assembly lines that comes off of and with the same ASIC. They just cut out the middle man and deliver the hardware directly to you. I've had WB switches running 24/7 for > 365 days with 0 component failures.
2) Infrastructure focused operating systems are not about presenting things as "magic". Yes there are abstractions, but the engineer should understand and be able to manipulate these abstractions. The end user should see the network as magic (as my mom views her iPhone and MAC), but not the engineer.
We are also considering WB for a few point solution roles (Cumulus/Dell, Pluribus, and a couple others) and we are hoping that they could eventually find their way into Production. Any experiences you can share would be helpful.
http://blog.ipspace.net/2014/10/cumulus-linux-in-real-life-on-software.html
I run Linux on my laptop. Though it is a Lenovo...
I.e. it all boils down to
- how much money do you want to spend on the equipment ?
- what kind of suppport do you need/get from whoever/whatever you're using (and how well do you trust that/depend on and are willing to try)?
- do you have skills (time/people) to do more or less on your own ?
Best practice - try/implement new thingies in small/controlled environment and then scale if they prove their value (which is used here as a VERY wide term).
And I agree, I see a lot of "fancy-gadget" owners preaching sth completely different. As if they've never heard of "practice what you preach".
Anyhow, time will tell...
Or as Heraclitus once said "The only constant in life is change".
And that is the beauty :) but that's another story
I like the cumulus linux example. Run that server OS as a network OS using server boxes and nics(to make a switch) or white box switches.
Didn't we do that with Novell Netware, 3Com, and Banyan with server OS and server nics years ago? We even had Netframe custom solutions until the performance of that collapsed model peaked thus requiring purpose built devices/silicon from Bay, Cisco, etc of that time separating server, routing and switching out of that single box. Yes the x86 platform and memory is much cheaper and faster today but so is our thirst for bandwidth.
So I will build a cheap network of white box x86 servers with standard nics and run my enterprise on it. uh okay. It is so cheap I can swap anything out at will without even troubleshooting.
History repeating itself with Moores Law in a different dimension. LOL!
There are going to be those special - read purpose built "bakes"- and if Cumulus gets the sauce right then they aren't quite white box and fall in with the rest of the purpose built, ASR, Junipter HW etc. I made reference to Cumulus for their cute video played it out that simply. a LinuxOS , nics and away we go.
Is that IOS-XE Linux kernel, since it is open sourced tweaked or changed for the hardware or do they just grab a "ubuntu like distro" lol and load it up on ASR HW and throw on some networking kernel modules?
Laptops have always had more deep customization and special hardware or custom hardware which requires more software integration.
Servers are typically sold without any software on them, and customers can install their own VMware, Windows or Linux on them to do what they want.
A more relevant question would be to go into any datacenter (including your own, Ivan!) and look at the servers and ask whether they are running some custom version of OS that came with the server hardware. In the old days, this _was_ the case (with Sun, SGI, HP servers, IBM servers, mainframes, DEC, etc.). I know you love the lessons of history!
Today, server hardware is interchangeable, and the key insight is not whether they come from some name-brand server company (HP, Dell, etc.) or white box vendor, but that the software and hardware are bought and managed separately, and that this model is far superior to the old one where vendors tried to sell you both.
Server companies are not selling you software, and they are not building you software, at least not enterprise-grade server OS's. Ask yourself why that is, and then ask why networking is fundamentally different to justify a perpetual connection between hardware and software.
If your apps are right then the hardware is going to matter a lot less then it has in days past.
The whole notion that there is something shoddy about the hardware or software vs. the legacy vendors is weird, given the quality of the software coming from them. Solaris was a way better engineered OS than anything coming from Cisco today, and Solaris is still gone today because Linux and Windows overtook it.
That is why server OS runs well on commodity server hw - it was designed for it.
Now on one hand you can build a network with all off the shelf x86 HW and NICs throw/compile a Linux distro on them for a routed or switched solution and it will operate consistency since all the nics and buffers et al. are the same(as long as you get the exact cards). There you go, but how well will it perform vs. purpose built network components and how much can you expand if you wanted to introduce other features? There are some of those large clusters using x86 commodity and in one way they are purpose built out of commodity HW/SW but in bulk scale to achieve a figure of merit at a lower cost than that of a smaller but more expensive solution. Some call it HPCC but some of those use specialized HW too.
I guess my point is now it is true that purpose built network HW and overbuilt network OS(read features you purchased but never use) does contribute to the cost but the decoupling via virtualization, SDN and the use of purpose built HW for those performance features but with non monolithic module Network OS so you can “bolt on” features when you need them provide a decent balance today.
Remember Novell NLMs, they had an IP one, SNA GW one et. al. – seems familiar.
You are not going to put a POS interface in an x86 box and run that as a core Linux router?
A trading floor network won’t do this.
Can you build an 802.11 wireless network this way too, yes but why would you. In 03, I used x86 as a low cost 802.11b 10mb PtP solution for a client that provided a link between buildings of a small business and it handled VoIP. I used commodity HW/SW off the shelf wifi nics, did the RF/Spectrum analysis, Fresnel zone testing, a couple of cheap directional(almost coffee/Pringles canned it too;) )
It worked and it worked well, it met its figure of merit on performance and cost but that was all you would get out of it, no scalability outside of upgrading nics to a higher 802.11 spec. or adding multiple PtP boxes which would get clumsy compared to IR or purpose built Wifi solutions.
Look at the Summary statement in one of Ivan's earlier posts about Brocade to see what I mean. There is a reason why whitebox is not the panacea.
http://blog.ipspace.net/2014/03/per-packet-load-balancing-interferes.html
TCO to headache ratio of whitebox will be better than that of the incumbents. Whitebox already wins in web-scale scenarios and the other use cases will be won soon enough since the pace of open source innovation will make them catch and overrun the incumbents.
--
Sent from my iPad
But to add fodder for your initial topic saw this on linked in today
Facebook just fired another big shot at Cisco — and dissed it a little, too
Read more: http://www.businessinsider.com/facebook-releases-6-pack-switch-in-shot-at-cisco-2015-2#ixzz3RaM1oKm0
http://www.businessinsider.com/facebook-releases-6-pack-switch-in-shot-at-cisco-2015-2?utm_source=linkedin-ticker&utm_medium=referral
Not just that, but I don't think your comparison of Juniper/Cisco being apple, and the Quantas being no one is valid. The value-add in disaggregating the hardware and software is that you got a platform (Debian Linux) that's actually more stable than the proprietary flavors still running on modern network hardware.
Not just that, but the chipsets running most of the modern Nexus gear (as an example) is the same Broadcom Trident 2 you find in the Quanta/Penguin switches.