Whitebox Hardware and Open-Source Software

One of my subscribers was interested in trying out whitebox solutions. He wrote:

What open source/whitebox software/hardware should I look at if I wanted to build a leaf-and-spine VXLAN/EVPN/BGP data center.

I don’t think you can get a fully-open-source solution because the ASIC manufacturers hide their SDK behind a mountain of NDAs (that strategy must make perfect sense – after all, it generated such awesome PR for NVIDIA). Anyway, the closest you can get (AFAIK) if you're a mere mortal is Cumulus Linux, and you just choose any whitebox hardware off their Hardware Compatibility List.

I also want to look into open source/white label for my firewalling needs, DMVPN over mGRE, and SD-WAN needs.

Apart from few firewall manufacturers touting their awesome ASICs nobody does firewalling in hardware, so all you need is an x86 server and an open-source firewall (suggestions welcome: please write a comment).

Modern open-source IPsec implementations already use AES-NI instruction set, so you could say you’re using whitebox hardware for your IPsec implementation even when you’re running OpenVPN on an x86 server.

DMVPN is proprietary as are all SD-WAN solutions so no luck there, although someone recently wrote about open-source SD-WAN solution. Had no time to dig deeper, and maybe they do have a high-quality open-source implementation, but their whitepaper is so buzzword-loaded that my BS detector experienced immediate overload.

More information

3 comments:

  1. Mellanox is developing an open-source driver for their Spectrum ASIC https://github.com/mellanox/mlxsw/wiki although I don't know of any NOSes that use it; I guess their traditional SDK is ahead on features.

    For open-source-ish NOSes I guess you're down to Cumulus which is a product or SONIC which may be "more open" but doesn't seem to be commercially supported. In PCs and servers it was a big deal to get a more advanced OS for $0, but switches are not free and most people don't just have them lying around so a free-but-unsupported product has less appeal.
  2. The ASIC NDA issue remains, but there is the Facebook et al's Open Compute Project (OCP) and Open Switching System (FBOSS). Interesting in it's own way if you're brave :)
  3. An underappreciate factor in whitebox switches is the platform layer. There is no standard for how to spin fans, poll SFPs or flash LEDs. Every switch requires device drivers for the platform, even without the ASIC. Cumulus (my employer) spends a lot of time on this. If you look at the ONL platform drivers it's almost all OCP switches, since the specs are public for people to build FOSS drivers for. If the specs aren't public then you have to work directly with the switch vendor to figure out how to turn the thing on in the first place. The boring stuff turns out to be the most complicated in whitebox land.
Add comment
Sidebar