ONIE and the Hammer of Thor

Someone left a comment on my Zero-Touch Provisioning post claiming how Big Switch Networks solved ZTP challenge using just IPv6 Link-Local Address and Neighbor Discovery instead of the complicated DHCP/TFTP/whatever sequence.

Here’s what he wrote:

They use ipv6 link-local addressing with a combination neighbor discovery to locate the controllers. The caveat being that the controllers need to be L2 adjacent with the switch mgmt ports that they hope to provision. Still...its better than anything DHCP related with all the options and image repos involved.

Somehow that didn’t sound right. After all, we don’t need DHCP to assign an IP address to the switch during zero-touch provisioning process (we could use IPv4 link-local addresses) but to point the switch to its software image and configuration file.

I tried to figure out what’s going on behind the scenes and failed. The Dell/VMware/BSN document he quoted spends exactly one paragraph on this topic without providing any details, and the documentation is still behind a regwall.

However, as Big Switch Networks works with whitebox switches using ONIE (PXE boot for switches), it was pretty easy to find out how you can get ONIE to work with IPv6 link-local addresses.

ONIE bootloader tries all the sane approaches first: local USB file system, URL passed in DHCP response (supporting HTTP, FTP and TFTP) and URL guessing based on DHCP options. If all that fails, it decides it’s time to use a bigger hammer (remember: never use brute force when a bigger hammer would do the trick):

  • Ping all neighbors using broadcast IPv4 ping and all-node multicast IPv6 ping;
  • For every neighbor responding to the ping, try to reach it over HTTP (port 80);
  • For every web server present on the local network, try two-dozen image names.

Yes, the approach works, but the hammer it uses would probably make Thor proud… and you better hope that:

  • The switches send ONIE requests only on their management port(s);
  • The management network is totally isolated from the user-facing infrastructure;
  • Someone hasn’t broken into one of your switches and started a web server on it (remember: they all run Linux these days) just for the giggles.

Failing either one of these assumptions could result in an interesting security challenge.

11 comments:

  1. It resembles me old good days from 90ties. Novell Netware, diskless PC with bootROM loading DOS (and later even Windows 3) from the Netware Server. Deployed tens Netware setups working without any admin emplyed on site. Zero administration of PCs....
  2. Thanks for the detailed explanations. It's seems to be a true Macgyver tool. In whatever situation you are in, it get's the job done.
  3. And why on earth would you have your controller L2 adjacent to the device booting up? Unless it's in a constrained DC environment, which makes the feat far less impressive.

    Wouldn't work so well for SD-WAN or campus networks where you want to template things from a central location...
    Replies
    1. BigSwitch makes software for data center fabrics, where you'd expect the switches to have a separate management network connection back to the controller (or whatever you want to call it). In smaller fabrics, the management network is indeed a L2 network (including QFabric IIRC).
    2. Sure. My point was that it's trivial to write something that works over a LAN and the same approach would not necessarily work over a WAN. So if you are going to have a more comprehensive ZTP, you will need something else, such as DHCP to solve the problem.
  4. so ONIE is developed by Bigswitch then, how weird does this blog gets.
    Replies
    1. Dear Anonymous. I reread what I wrote and one of us has serious reading comprehension problems. I would suggest you stop reading my blog posts. Obviously they annoy you way too much, making you irrational.
  5. http://www.opencompute.org/wiki/Networking/ONIE

    Created by Cumulus Networks, Inc. in 2012, the Open Network Install Environment (ONIE) Project is a small operating system, pre-installed as firmware on bare metal network switches, that provides an environment for automated operating system provisioning.
  6. ONIE User Guide:
    https://opencomputeproject.github.io/onie/user-guide/

    ONIE Source Code:
    https://github.com/opencomputeproject/onie/
  7. Anonymous. Ivan did not say ONIE was developed by BigSwitch. He said BigSwitch works on whitebox hardware. The whitebox hardware uses ONIE.
  8. I remember the first time I watched a switch ONIE boot. The big switch stuff is fantastic. I’m just so over caring how my datacenter fabric works under the hood. All I need is the MAC address of the management port, I enter that in and plug the switch in. Minute or so later I start plugging in servers and move on with my datacenter life lol. Don’t get them wrong, they aren’t going to do your campus/WAN/routing or anything like that, but for a datacenter switching fabric it’s nearly perfect.
Add comment
Sidebar