Run Well-Designed Experiments to Learn Faster

I know that everyone learns in a slightly different way. Let me share the approach that usually works well for me when a tough topic I’m trying to master includes a practical (hands-on) component: running controlled experiments.

Sounds arcane and purely academic? How about a simple example?

A week ago I talked about this same concept in the Building Network Automation Solutions online course. The video is already online and you get immediate access to it (and the rest of the course) when you register for the next live session.

A network engineer enrolled into my network automation online course was trying to integrate external Python script with Ansible and asked this question in the course Slack team.

I'm trying to call a Python script from an Ansible task and want to pass the script's result to an Ansible variable (dictionary) in the playbook so I can then loop through it.

I had a pretty good idea how to do it, but wanted to test it first. Here’s how I ran my experiment.

You might not care about Ansible at all, which is perfectly fine. I just needed a real-life example to illustrate my ideas.

Start in a clean environment

Nothing is more frustrating than someone saying “could you take a look at this” and showing you zillion versions of the (almost) same script intermixed with a ton of other irrelevant files.

Start every experiment in a separate (empty) directory. It will make your job easier because you’ll know what you’re working on. You will also get quicker responses when asking for help.

Reproduce the behavior using a bare minimum of simple components

Don’t try to bolt new things onto an existing convoluted script (or playbook) – it will be pretty hard to figure out what’s causing the unexpected behavior, and the experiment might run for a long time before failing.

Having a minimal fast-executing environment results in more controlled experiments, more reliable results, and faster progress.

Do note that this is just the opposite of google-and-paste method favored by some IT practitioners.

It also helps if every component you use is as simple as possible. In my case I decided to:

  • Create a minimal playbook with a single task that would test how Ansible treats results returned by an external script;
  • Skip Ansible inventory and use localhost;
  • Have JSON data in a text file instead of running a Python script (remember: we’re not testing whether the Python script works but how Ansible treats the output).

Here’s the Ansible playbook I used:

- hosts: localhost
  gather_facts: no
  - shell: cat data.json
    register: shell_output

I knew that I’d have to register the results of the shell task as an Ansible fact, but had no idea in what format those results would be.

Here’s my JSON data file:

{ 'a':'b' }

Check your data before running the experiment. One of the most frustrating experiences you could have is trying to figure out how something works and failing for hours… only to discover your input data was wrong (or you made a stupid syntax error or…).

If you’re fluent in JSON you might have experienced a facepalm moment while looking at my JSON data. I used jq to check it and got this:

$ jq . data.json
parse error: Invalid numeric literal at line 1, column 6

It makes no sense to progress with your experiment till you have valid input data. A quick check on JSON syntax told me I should have used double quotes (trust me, you can waste hours on a stupidity like this).

Here’s the correct version of my JSON data file:

{ "a":"b" }

Make small steps

Next, I ran the playbook in verbose mode to see what the shell module returns:

$ ansible-playbook pb.yml -v

TASK [command] **********************************
changed: [localhost] => {"changed": true, "cmd": "cat data.json", 
  "delta": "0:00:00.003140", "end": "2017-10-04 12:09:43.255971", 
  "failed": false, "rc": 0, 
  "start": "2017-10-04 12:09:43.252831", 
  "stderr": "", "stderr_lines": [], 
  "stdout": "{ \"a\":\"b\" }", 
  "stdout_lines": ["{ \"a\":\"b\" }"]}

It’s pretty obvious that the shell module returns a string (stdout) containing whatever the bash command has printed. Time to convert that string into parsed data. Fortunately, I could use from_json filter in Ansible to do that.

Next version of my playbook:

- hosts: localhost
  gather_facts: no
  - shell: cat data.json
    register: shell_output
  - set_fact: json_data={{shell_output.stdout|from_json}}

Running that playbook in verbose mode produced something that looked like the data structure I was looking for. Final step: use debug to print out the actual value of the json_data variable.

$ ansible-playbook pb.yml

TASK [debug] ****************************************
ok: [localhost] => {
    "json_data": {
        "a": "b"

Next steps

After figuring out how to get JSON data into an Ansible fact, I could explore further:

  • Use jq to check that the Python script returns properly-formatted JSON data;
  • Replace cat with the Python script;
  • Check that I get the desired data in Ansible;
  • Use that data in my playbook.

Sounds interesting?

You’ll be solving tons of interesting challenges like the one above in Ansible for Networking Engineers or Building Network Automation Solutions online courses.

You can also access most of the materials from the Ansible online course with subscription.

This blog post was initially sent to the subscribers of my SDN and Network Automation mailing list. Subscribe here.


  1. I got you're making the point of how to approach challenges in technology and I fully agree with that approach. Start simple and grow.
    Just like to add that depending on the python script you might even want to include it as an ansible module, which seems dead easy.
    1. For someone who just started getting familiar with network automation (and Python), that might be way too complex (but otherwise I agree with you). Also, it's simpler to troubleshoot a simple Python script than an Ansible module.
  2. -- Method in madness --

    Your method is fine but the Ansible tool/language seems pretty mad to me.
    The simple task of reading a file is complicated beyond measure.
    Maybe Ansible has other qualities but even Fortran, invented 60 yars ago, has simpler input/output syntax!!
    1. Repeat after me:

      Ansible is NOT a programming language
      Ansible is NOT a programming language
      Ansible is NOT...

  3. Is there a debugger for ansible?
    1. Yes, see
  4. This is common development practice. It is well explained in the (online free) book Think Python, which I found very interesting in this regard. The "debug" paragraph on each chapter has real value and the book is worth reading even for people who are writing small scripts from time to time.

    Here is an extract of the book:

    A development plan

    A development plan is a process for writing programs. The process we used in this case study is “encapsulation and generalization.” The steps of this process are:

    1. Start by writing a small program with no function definitions.
    2. Once you get the program working, encapsulate it in a function and give it a name.
    3. Generalize the function by adding appropriate parameters.
    4. Repeat steps 1–3 until you have a set of working functions. Copy and paste working code to avoid retyping (and re-debugging).
    5. Look for opportunities to improve the program by refactoring. For example, if you have similar code in several places, consider factoring it into an appropriately general function.

    This process has some drawbacks—we will see alternatives later—but it can be useful if you don’t know ahead of time how to divide the program into functions. This approach lets you design as you go along.
    1. "This is common development practice." << Agreed. Unfortunately, like with common sense, it's somewhat rare in the wild ;))

      Thanks for the book reference - will add it to "recommended self-study materials"
Add comment