XML-to-JSON Information Loss, Cisco Nexus OS Edition

Last week I wrote about the interesting challenges you might encounter when using data generated by a Junos device in an Ansible playbook. Unfortunately it’s not just Junos – every system built around XML-based data structures might experience the same issues, including Cisco Nexus OS.

To be fair to Ansible developers: it’s not an Ansible problem, the problem is caused by fundamental incompatibility between XML and JSON encodings, and the naive use of standard XML Python libraries. It’s just that engineers who might stumble upon that problem commonly use Ansible.

TL&DR: there’s a summary at the end of this blog post, including the correct way of doing things on Nexus OS.

Let’s examine the XML document created by a show vlan command on a newly configured Cisco Nexus switch. Focus on the ROW_vlanbrief tag:

XML document describing a single VLAN configured on a Nexus switch
switch# show vlan | xml
<?xml version="1.0" encoding="ISO-8859-1"?>
<nf:rpc-reply
  xmlns="http://www.cisco.com/nxos:1.0:vlan_mgr_cli" 
  xmlns:nf="urn:ietf:params:xml:ns:netconf:base:1.0">
 <nf:data>
  <show>
   <vlan>
    <__XML__OPT_Cmd_show_vlan___readonly__>
     <__readonly__>
      <TABLE_vlanbrief>
       <ROW_vlanbrief>
        <vlanshowbr-vlanid>1</vlanshowbr-vlanid>
        <vlanshowbr-vlanid-utf>1</vlanshowbr-vlanid-utf>
        <vlanshowbr-vlanname>default</vlanshowbr-vlanname>
        <vlanshowbr-vlanstate>active</vlanshowbr-vlanstate>
        <vlanshowbr-shutstate>noshutdown</vlanshowbr-shutstate>
        <vlanshowplist-ifidx>...</vlanshowplist-ifidx>
       </ROW_vlanbrief>
      </TABLE_vlanbrief>
      <TABLE_mtuinfo>
       <ROW_mtuinfo>
        <vlanshowinfo-vlanid>1</vlanshowinfo-vlanid>
        <vlanshowinfo-media-type>enet</vlanshowinfo-media-type>
        <vlanshowinfo-vlanmode>ce-vlan</vlanshowinfo-vlanmode>
       </ROW_mtuinfo>
      </TABLE_mtuinfo>
     </__readonly__>
    </__XML__OPT_Cmd_show_vlan___readonly__>
   </vlan>
  </show>
 </nf:data>
</nf:rpc-reply>
]]>]]>

Notes:

  • I removed the contents of vlanshowplist-ifidx tag to make the printout fit the column width.
  • Noticed the weird ]]>]]> sequence at the end of the XML document? That’s NETCONF message delimiter sequence that SHOULD NOT be in an XML printout – no decent XML parser would parse what you got from a Nexus OS show | xml printout.

Configure another VLAN, repeat the show vlan command and examine the resulting XML document. You’ll notice two instances of the ROW_vlanbrief tag (if you read my previous blog post on this topic you probably know where this is going 😉)

XML document describing a two VLANs configured on a Nexus switch
switch# conf t
Enter configuration commands, one per line. End with CNTL/Z.
switch(config)# vlan 2
switch(config-vlan)# end
switch# show vlan | xml
<?xml version="1.0" encoding="ISO-8859-1"?>
<nf:rpc-reply
  xmlns="http://www.cisco.com/nxos:1.0:vlan_mgr_cli"
  xmlns:nf="urn:ietf:params:xml:ns:netconf:base:1.0">
 <nf:data>
  <show>
   <vlan>
    <__XML__OPT_Cmd_show_vlan___readonly__>
     <__readonly__>
      <TABLE_vlanbrief>
       <ROW_vlanbrief>
        <vlanshowbr-vlanid>1</vlanshowbr-vlanid>
        <vlanshowbr-vlanid-utf>1</vlanshowbr-vlanid-utf>
        <vlanshowbr-vlanname>default</vlanshowbr-vlanname>
        <vlanshowbr-vlanstate>active</vlanshowbr-vlanstate>
        <vlanshowbr-shutstate>noshutdown</vlanshowbr-shutstate>
        <vlanshowplist-ifidx>...</vlanshowplist-ifidx>
       </ROW_vlanbrief>
       <ROW_vlanbrief>
        <vlanshowbr-vlanid>2</vlanshowbr-vlanid>
        <vlanshowbr-vlanid-utf>2</vlanshowbr-vlanid-utf>
        <vlanshowbr-vlanname>VLAN0002</vlanshowbr-vlanname>
        <vlanshowbr-vlanstate>active</vlanshowbr-vlanstate>
        <vlanshowbr-shutstate>noshutdown</vlanshowbr-shutstate>
       </ROW_vlanbrief>
      </TABLE_vlanbrief>
      <TABLE_mtuinfo>
       <ROW_mtuinfo>
        <vlanshowinfo-vlanid>1</vlanshowinfo-vlanid>
        <vlanshowinfo-media-type>enet</vlanshowinfo-media-type>
        <vlanshowinfo-vlanmode>ce-vlan</vlanshowinfo-vlanmode>
       </ROW_mtuinfo>
       <ROW_mtuinfo>
        <vlanshowinfo-vlanid>2</vlanshowinfo-vlanid>
        <vlanshowinfo-media-type>enet</vlanshowinfo-media-type>
        <vlanshowinfo-vlanmode>ce-vlan</vlanshowinfo-vlanmode>
       </ROW_mtuinfo>
      </TABLE_mtuinfo>
     </__readonly__>
    </__XML__OPT_Cmd_show_vlan___readonly__>
   </vlan>
  </show>
 </nf:data>
</nf:rpc-reply>
]]>]]>

While Cisco Nexus OS uses XML internally, it can also do an automatic XML-to-JSON conversion – just add | json to the show command. When used on the show vlan command, the JSON data structure returns a list of configured VLANs. As expected, the ROW_vlanbrief element contains a list of VLANs.

JSON object describing two VLANs configured on a Nexus switch
$ netlab connect c_nxos "show vlan | json" | jq
{
  "TABLE_vlanbrief": {
    "ROW_vlanbrief": [
      {
        "vlanshowbr-vlanid": "1",
        "vlanshowbr-vlanid-utf": "1",
        "vlanshowbr-vlanname": "default",
        "vlanshowbr-vlanstate": "active",
        "vlanshowbr-shutstate": "noshutdown",
        "vlanshowplist-ifidx": "..."
      },
      {
        "vlanshowbr-vlanid": "2",
        "vlanshowbr-vlanid-utf": "2",
        "vlanshowbr-vlanname": "VLAN0002",
        "vlanshowbr-vlanstate": "active",
        "vlanshowbr-shutstate": "noshutdown"
      }
    ]
  },
  "TABLE_mtuinfo": {
    "ROW_mtuinfo": [
      {
        "vlanshowinfo-vlanid": "1",
        "vlanshowinfo-media-type": "enet",
        "vlanshowinfo-vlanmode": "ce-vlan"
      },
      {
        "vlanshowinfo-vlanid": "2",
        "vlanshowinfo-media-type": "enet",
        "vlanshowinfo-vlanmode": "ce-vlan"
      }
    ]
  }
}

Notes

  • As before, I removed the contents of vlanshowplist-ifidx tag.
  • While Nexus OS displays nicely formatted XML document, its JSON printout is a mess. I decided to send it through jq so you’ll be able to see what’s going on without parsing levels of curly brackets by hand.
  • As Thomas pointed out in a comment, you could use | json-pretty filter and get a decent-looking JSON printout from the device itself.
  • netlab connect is a netlab command that extracts host IP address and SSH parameters from Ansible inventory and uses them to connect to the device or execute a SSH command.

But what happens when you execute the same command on a vanilla switch with a single VLAN? Here’s what you get back:

JSON object describing two VLANs configured on a Nexus switch
$ netlab connect c_nxos "show vlan | json" | jq
{
  "TABLE_vlanbrief": {
    "ROW_vlanbrief": {
      "vlanshowbr-vlanid": "1",
      "vlanshowbr-vlanid-utf": "1",
      "vlanshowbr-vlanname": "default",
      "vlanshowbr-vlanstate": "active",
      "vlanshowbr-shutstate": "noshutdown",
      "vlanshowplist-ifidx": "..."
    }
  },
  "TABLE_mtuinfo": {
    "ROW_mtuinfo": {
      "vlanshowinfo-vlanid": "1",
      "vlanshowinfo-media-type": "enet",
      "vlanshowinfo-vlanmode": "ce-vlan"
    }
  }
}

What you get back is a ROW_vlanbrief dictionary, not a list of VLANs, proving that Nexus OS uses the same naive way of converting XML to JSON as jxmlease library used by junos_command Ansible module.

Summary

  • Cisco Nexus OS uses XML data structures internally;
  • Those data structures could be displayed as a result of a show command (with some extra garbage at the end to ensure you’ll have fun parsing it)
  • The same data structures could also be displayed as a JSON object… but you’ll consistently get a dictionary instead of a list with one element
  • You can ask NXAPI to return a list instead of a dictionary if you use cli_array method instead of cli method (see below).
  • If you’re developing your automation solution in Python, or are willing to use a custom Jinja2 filter, you could use the fact that everything that should be a list has a name starting with ROW_ and turn one-item dictionaries into lists. See this blog post by Christopher Hart for details.
  • Ansible nxos_command module does not parse XML printout (like junos_command does… not that it helps). The only way to get structured data from Nexus OS is to generate a JSON document on the switch.

Long story short: Have fun trying to deal with things that might have one or more instances. The path to network automation is truly littered with broken glass.

Fix: Use NXAPI with cli_array Method

Shortly after the blog post was published Gerard Sheehan contacted me and provided a solution to this conundrum… assuming you can use NXAPI instead of CLI


The primary programmatic interface into Cisco NX-OS switches is NXAPI, and this is what Cisco directs all customers and partners to use. NXAPI CLI provides access to the same CLI commands and output as via the terminal.

Several years ago, in NXAPI Cisco added a new method that will return a single item as a single item within a list. The new method is called cli_array, and the various methods are documented here. For customers who want the existing structure and format, we still support that via cli method. All of this can be seen via the NXAPI sandbox.


Thank you!

Interesting Opinions

Not surprisingly, the blog post triggered some interesting opinions, from “This won’t be an issue on a Nokia SR OS that took an effort to make YANG a first-class citizen” by Roman Dodin to “it’s not a big deal, here’s how you fix it” quickly countered byI disagree, it is an api - it must produce deterministic output

What I particularly liked was this comment made by Johann Lo on LinkedIn:

This is why until vendors have up-to-standards APIs across the board you’re always going to end up diving down rabbit holes and reinventing the wheel doing bespoke ground-up automation, esp. cross vendor. You’re just paying in developer time not for product costs. I mean look at the state of affairs, we’re still getting blog post after blog post about logging into something and extracting a show command into structured data… whoopee.

This is not to say its not worth it…. but it’s not the silver bullet that people assume it is, esp. coming from a cloud background where its basically easy mode for automation.

Stuff like pyATs/genie are going to go a long way yes but even then what happens with the rest of your non-IOS stack?

The fact is that most enterprises still seem to insist on throwing net-engs at development problems instead of the other way round. Some of us may turn out to be great coders, but most of us will produce mediocre hacky code using much more time than a real dev who’s been coding full time whilst we were wrangling BGP/OSPF/STP. And then we’re all reinventing the wheel together, a million slightly different ways to get ansible running as a glorified looping machine over a bunch of json text files.

Revision History

2022-08-28
Use the new netlab CLI commands
2021-03-17
Added a pointer to a Python-based workaround
2021-01-22
Added cli_array NXAPI method
2021-01-21
Added information about json-pretty filter and a few interesting opinions from Twitter and LinkedIn.

Blog posts in this series

Add comment
Sidebar