Automation Win: MPLS/VPN Service Deployment

I always encourage the students attending the Building Network Automation Solutions online course to create solutions for problems they’re facing in their networks instead of wasting time with vanilla hands-on assignments.

Francois Herbet took the advice literally and decided to create a solution that would configure PE-routers and create full-blown device configurations for CE-routers.

He’s one of those people that thoroughly document whatever they’re doing, so you can track how he progressed through the course in his GitHub repository:

  • Getting started: he built a large lab in EVE-NG (here’s the answer for everyone asking “Can I use EVE-NG to build labs in your course?”)
  • Easy wins: collect configurations and show printouts. Collect NAPALM facts.
  • Data model: this is how he structured information about a customer order (probably coming from some back-end ordering system).
  • Deploy the services: creating IOS and IOS-XR configurations for PE- and CE-routers. Deploying final configurations to PE- and CE-routers.
  • Logging and Testing: He decided to validate input data and wrote unit tests that checked the validation code. He also added change logging to his playbooks and performed a number of deployment validation tests.
  • Going into production: The concepts Francois mastered during the course are already in production. You won’t be able to see the final code for obvious reasons, but you can get an idea of how far he got just a few months after he started the course and how well things are working in real life.

The awesome end result:

When field techs go on site to complete installation, we can bring site live for our customer 10 minutes after CPE is powered on.

I asked him if he’d have time for a beer during Cisco Live Europe but unfortunately he was too busy:

Have bunch of assert tasks to build to make my sanity checks more robust, and several hundred sites in my current pipeline where my playbook based on the one I've build during the automation course will handle fully automated provisioning, service configuration + sanity checks...

I’d say that’s a sweet problem to have ;) … and a great answer to the question “What will happen to my job now that we’re automating everything?

Latest Update from Francois

This is what he sent me a few days ago:

I’m now almost at full speed: over the last 2 weeks, I have been able to deliver (alone) ~40 dual homed sites (80 CPEs) using Ansible.

I built a cron job that checks for management loopback interface reachability every 5 minutes and runs all-in-one playbook to check/configure PE/CPE, automate final tests and sends confirmation email to internal teams and customer with tech details so they can start monitoring the devices.

I just have to troubleshoot failed ones (based on Ansible logs I get via emails) and answer field techs’ calls to say most of the time “Thanks mate, everything’s done! You can go back home!”.

The only thing I wish now is that all ISPs I have to work with would be as agile as the one Francois works for ;)

3 comments:

  1. Awesome build! Any idea how he/people in general handle the chicken-egg problem? How does the CPE become reachable from the Ansible host to finalize the config? A Cisco zero install method? Generiek DHCP-kinda config?
    Replies
    1. Hi Stefan, thanks for your comment! ;-)
      Basically, field technician is given a MGMT LB + /30 to set on WAN interface, He has to set that on site + a default route to remote WAN IP + local authentication and here we go...
      As we have ethernet support in most the case, that's not something too difficult for them.
      Default WAN interface is always the same depending on CPE model...
      i'll try to automatically generate boot template for them in the next few months... but it's step by step right now!
  2. Great Article Francois!
    Thank You for sharing ansible yml files :)
    I would have to lab it use it eve-ng as well. Still far behind compering what You have accomplished.
Add comment
Sidebar