After finishing the network automation part of a recent SDN workshop I told the attendees “Vote with your wallet. If your current vendor doesn’t support the network automation functionality you need, move on.”
The Pass/Fail information included below was collected to the best of my knowledge with extensive help from Jason Edelman, Nick Buraglio, David Barroso and several Brocade engineers (THANK YOU!). If you feel the information is incorrect, please point me to public documentation that proves me wrong. Also, do provide information on other platforms.
Update 2016-10-27: Cisco IOS structured configuration is almost OK (moved to Pass (mostly))
Update 2017-03-04: Cisco IOS XE release 16 generates properly-nested configuration elements. Moved to Pass. Also added rollback-on-error capability of IOS XE release 16.
Programmable interface (API)
The device MUST have an on-device programmable API (NETCONF or REST) that allows an external script to:
- Get device configuration
- Get operational data
- Change device configuration.
I don’t want to hear about “solutions” that insert layers of kludges between my script and the device I want to manage. If I can’t access the device itself using NETCONF or REST I’m no longer interested. After all, my calendar is showing 2016.
Pass: Most networking vendors, at least in recent software releases.
Fail: List your grievances in the comments ;)
Structured operational data
The device MUST return operational data as structured data (JSON or XML format) not as text printouts wrapped in XML or JSON envelopes.
I had enough of screen scraping in the 30 years I had to deal with networking devices. I don’t want to write another Expect script or TextFSM definition. My calendar is still showing 2016.
Pass: Junos, Nexus OS, Arista EOS, Brocade VDX, ALU/Nokia
Fail: Cisco IOS
Nice try: Cisco IOS XE with REST API (it returns a minimalistic set of operational data, see also ''feature parity'' below).
Device configuration in structured format
The device SHOULD return its configuration in structured format (JSON or XML) with meaningful structure (for example, ACL lines should be within the ACL).
I don’t know why I should write another configuration scraping program to figure out what BGP neighbors a device has if I could do the same thing with a simple walk down the return object. I had enough Perl Regexps for one life.
Pass: Junos, Brocade VDX, ALU/Nokia, Cisco IOS XE release 16.
Mostly there: Cisco IOS and IOS-XE (prior to release 16).
Atomic configuration changes
Changes to device configuration MUST be atomic, more so if the device supports NETCONF – either all the submitted changes are accepted or none is.
I really don’t care if I can get that done in a NETCONF session with commit capability or as a single huge REST call, but I don’t want to be cut off the box once again because the box accepted only half the ACL.
Pass: Junos, IOS XR, probably Arista EOS
Almost: Cisco IOS XE. REST interface is atomic within a single call, as is NETCONF implementation in release 16.x which implements rollback-on-error.
Fail: Cisco IOS, Nexus OS, Brocade VDX
The device MUST support rollback to a previous configuration.
If I made a mistake, I want to be able to go back to a previous configuration without spending hours hand-crafting the differences between the mess I made and the configuration that worked before I started messing it up.
Pass: Junos, IOS XR, Arista EOS, Cisco IOS, Nexus OS, ALU/Nokia
Fail: Brocade VDX
The device MUST support replacing current configuration with a new configuration without a reload.
Sometimes I really don’t want to waste my time calculating the differences that have to be made to get the device to do what I want, particularly when I create the whole configuration with a template.
Pass: Junos, IOS XR, Arista EOS, Cisco IOS/XE
Fail: Nexus OS, Brocade VDX
The device SHOULD be able to create a list of configuration commands needed to transform one configuration into another.
It’s great if you can point out the differences between two configurations to the engineer who has to approve the change. Oh, and I’m looking for the list of commands to get from A to B. I can run a diff on Linux myself.
Pass: Junos, Cisco IOS
Fail: Most everyone else. Many platforms use standard Linux diff instead of considering configuration context.
For example, if I change BGP AS number, I’d like to see no router bgp A followed by router bgp B followed by whole BGP configuration.
Support for industry-standard models
The device SHOULD support industry-standard configuration data models (IETF and/or OpenConfig).
We waited long enough to get them. I don’t want to wait another decade for the vendors to implement them.
Partial: IOS XE (IETF), IOS XR (OpenConfig)
Fail: pretty much everyone else
Paraphrasing Ron Broersma: All functionality requested in the RFP must be fully supported by the device API and meet the above requirements.
I probably forgot a few critical requirements. Please list them in the comments.