Ansible Set Operations Do Not Preserve List Order

Here’s another Ansible quirk, this time caused by Python set behavior.

When I created the initial device configuration deployment playbook in netlab, I wanted to:

  • Be able to specify a list of modules to provision.1
  • Provision just the modules used in the topology and specified in the list of modules.

This allows you to use netlab initial to deploy all configuration modules used in a lab topology or netlab initial -m ospf to deploy just OSPF while surviving netlab initial -m foo (which would do nothing).

We covered the first part of this saga in Precedence of Ansible Extra Variables:

- set_fact:
    mod_select: "{{ modlist.split(',') if modlist is defined else netlab_module }}"

That would generate a list of modules, either those specified on the command line with the modlist extra variable or all modules used in the topology (specified in the netlab_module inventory variable).

It’s worth noting that the order of configuration deployment matters. For example, you have to configure VLANs before OSPF to have the VLAN interfaces that can be specified in the OSPF process.2 To get that done, I decided to create a loop that would iterate over all modules specified in the lab topology and in the mod_select variable (I had to use this approach instead of just iterating over device modules due to another Ansible quirk):

- include_tasks: "tasks/deploy-module.yml"
  loop: "{{ netlab_module|intersect(mod_select) }}"
  loop_control:
    loop_var: config_module

Now for the fun part: this approach works most of the time3, but when I combined DHCP and OSPF modules, their order was swapped approximately half of the time (and yes, it’s non-deterministic, which makes it even more annoying).

It turned out the root cause was the intersect filter. While all the examples in the Ansible documentation imply that the order of elements is preserved, in reality it is not. The intersect filter turns both arguments into sets, does the intersection operation, and returns a list. Python documentation is quite clear: sets are unordered collections, and so we could get the elements in the intersection list in a completely different order.

Workaround: replace the intersect filter with a when condition:

- include_tasks: "tasks/deploy-module.yml"
  loop: "{{ netlab_module }}"
  when: config_module in mod_select
  loop_control:
    loop_var: config_module

On a totally unrelated train of thought: isn’t it great that we have enough time to waste half the Sunday morning chasing such gremlins?


  1. The default value is all modules used in the lab topology. ↩︎

  2. In case you’re curious: netlab modules specify which modules have to be configured before them, and netlab uses that information to do a topological sort and store the results in the netlab_module Ansible inventory variable. ↩︎

  3. OK, it works all the time 90% of the time 🤪 ↩︎

1 comments:

  1. Sadly the quality of Ansible hasn’t progressed in the direction that I had hoped…

    Replies
    1. This time, it's hard to blame them -- the filters are clearly marked as "set operations" and that would imply they're using sets.

      Expecting a typical server/network admin to know that Python sets are unordered is a different story 🤷‍♂️

Add comment
Sidebar