Every time I have a network automation presentation (be it a 2-day workshop or a 45 minute keynote) I get the same question afterwards: “How do we deal with exceptions?”

The correct answer is obvious: “there should be no exceptions, because one-offs usually cost you more than you earn with them,” but as always the reality tends to intervene.

There’s also the interesting challenge of rolling out pilot services: you don’t want to start modifying your orchestration platform until you’re pretty sure the new service will sell, which means you cannot provision the service, so it will never sell. A perfect Catch-22.

However, Rok Papež (a great engineer working on network automation within ARNES – Slovenian academic ISP) described a nice workaround while we chatted after my talk at recent SINOG event: they store the non-automated part of the device configuration (configuring the pilot services) in an extra field in their configuration management database, and append it to the configuration generated from device/service data model through standardized templates.

Now imagine you go one step further: instead of storing non-standard parts of device configuration your database contains configuration snippets for every non-standard (pilot or exception) user/service/device triplet. This approach makes it extremely easy to:

• Identify all pilot services and all exceptions;
• Identify which users have non-standard services;
• Verify whether a user has non-standard configuration before you start the troubleshooting process (and maybe escalate the ticket before wasting time trying to figure out what the deployment engineer had in mind);
• See how the engineers configured those services on individual devices before trying to configure a similar solution for the next customer;
• Verify that the orchestration system generates equivalent per-service configuration once the pilot service becomes part of the standard offering;
• Clean up non-standard parts of the configuration once you migrate the users of pilot services to new standardized services.

Thank you, Rok! You gave me a reasonable answer to a pretty tough challenge ;)