Smart NICs and Related Linux Kernel Infrastructure

A while ago we did a podcast with Luke Gorrie in which he explained why he’d love to have simple, dumb, and easy-to-work-with Ethernet NICs. What about the other side of the coin – smart NICs with their own CPU, RAM and operating system? Do they make sense, when and why would you use them, and how would you integrate them with Linux kernel?

We discussed these challenges with Or Gerlitz (Mellanox), Andy Gospodarek (Broadcom) and Jiri Pirko (Mellanox) in Episode 99 of Software Gone Wild.

Even though all three guests work for a NIC vendor, we quickly switched from the hardware aspects to the really interesting part: how do you make these NICs work, how do you control them, and how do you integrate them with host Linux kernel and its TCP/IP stack.

If you want to know even more about these topics after listening to the podcast consider attending NetDev conference (the next one is in Prague and starts on March 20th 2019), and if you want to know more about Amazon’s solution watch their Nitro presentations.

Guest Bios

Or Gerlitz works as Linux kernel developer since 2005, initially dealing with RDMA and storage and later happily moved to networking. He is an active contributor and sub-maintainer for the mlx5 upstream NIC driver, with his focus over the last years being the push for NIC HW e-switches to offload a proper kernel networking model used by SW v-switches serving VMs and Containers.

Andy Gospodarek is an Engineer/Architect at Broadcom and has been working with the Linux kernel networking since 2002. He has extensive experience with the architecture and implementation of hardware offload and acceleration on NICs, Network ASICs during his time at Red Hat, Cumulus Networks, and now Broadcom. Recently he has been exploring the value of control and dataplane offload to ARM-based SmartNICs in datacenter servers.

Jiri Pirko has been hacking Linux kernel since 2005, always with a focus on networking plumbing. Frustration caused by the state of bonding driver code caused him to do a complete rewrite called team driver. Another frustration caused by SDK binary blobs controlling switch ASIC lead him to come up with the idea of FOSS Linux switch solution and advocate it. Currently, he works on mlxsw (Mellanox Spectrum ASIC driver) and collaborates with other people on various Linux kernel networking plumbing projects, mainly connected with devlink and TC.

1 comments:

  1. So people realised software packet switching is slow and want to offload it to the hardware. History repeats itself. ;-)

    Actually software packet switching is not that slow. For many server-based workloads it may be "good enough" to overprovision 2 CPU cores more for network handling than to buy a smart NIC. I would like to see benchmarks and pricing.

    We do 100Gbit DDoS scrubbing on x86 platform purely in software and try not to rely on NIC advanced features. I second Luke's "dumb NIC" idea because it reduces complexity.
Add comment
Sidebar