Your browser failed to load CSS style sheets. Your browser or web proxy might not support elliptic-curve TLS

Building network automation solutions

6 week online course

reserve a seat
back to overview

Network Infrastructure as Database

A while ago I wrote about the idea of treating network infrastructure (and all other infrastructure) as code, and using the same processes application developers are using to write, test and deploy code to design and implement networks.

That approach clearly works well if you can virtualize (and clone ad infinitum) everything. We can virtualize appliances or even routers, but installed equipment and high-speed physical infrastructure remain somewhat resistant to that idea. We need a different paradigm, and the best analogy I could come up with is a database.

Imagine an organization that needs a large transactional database to run its business, and requires the frontline operators to modify data in that database with SQL statements (INSERT, UPDATE, DELETE). I haven’t heard of anyone being even remotely that stupid – and yet that’s how most of us run our networks. Time to take another lesson from application development world.

It’s obvious that one should interact with a production database only through well-defined and thoroughly tested transactions (whether you call them transactions, application programs or web scripts is irrelevant). Direct CLI (SQL) access to the database should be either prohibited or limited to the absolute minimum.

Likewise, we should make changes to our network devices with well-defined transactions (example: CREATE VLAN or REMOVE VLAN). These transactions (call them scripts, playbooks or recipes if you wish) should be treated like any application code, and tested in a lab environment before being allowed to touch the production network.

And now for the usual set of objections:

But I don’t trust scripts. Fantastic. Now tell me: is it better to rely on something that is tested and repeatable, or doing configuration changes on the fly hoping for the best?

How do I know it will work? Welcome to the world of software development. Write code, include unit and integration tests, test everything, deploy in a pilot environment, and finally in production. Lather, rinse, repeat.

But it might mess up my configurations. And you think nobody ever messed up their database? We have backups for a good reason – and most networking gear has some configuration versioning and rollback mechanisms.

But it might crash my switch. It might. Database systems have their own bugs, and I’m positive you were able to crash some databases with invalid data or weird requests a few decades ago… but guess what: unless we start screaming at the vendors, and vote with our wallets, nothing will change.

What about maintenance windows? We have to use maintenance windows in current networks because we can’t trust the manual configuration process. When was the last time you could do your e-banking transaction only at 2AM on a Sunday morning? Once everyone starts trusting the network configuration transactions, you’ll be able to use them at any time.

Have you encountered any other objections? Write a comment!

Finally, you might decide that your network isn’t big enough to warrant a network automation solution (or that you don’t have the manpower to work on it). The next best thing you might do is decoupling: move the dynamic parts of your network infrastructure into the virtual world where you can treat is as code, and keep the physical part of your network infrastructure as static as possible.

More information

Check out the SDN resources on ipSpace.net and listen to the Software Gone Wild podcast.

11 comments:

  1. I won't have a job if everything is automated! I really hate this one. I also hate when it isn't verbalized, but you see the wheels turning and coming to this conclusion inappropriately. Perhaps those folks shouldn't have a job if they don't see the advantages.

    ReplyDelete
    Replies
    1. Yes, being worried about how you are going to provide for your family in the future when it seems that the skills you've spent 10 years building aren't needed anymore is completely unnatural. They should probably just fire themselves right now. /sarcasm.

      The goal of "SDN" isn't to make life easier or "empower the network engineer" to do other things than to create vlans. While that is part of it, the goal is to make the skill set obsolete so you don't have to spend money on those people anymore. The only question is if it'll work to the extent that some people think.

      To "hate" someone for having these concerns only tells me that you completely lack empathy.

      Delete
    2. Of course it's a natural reaction if you think that your livelihood is going to be affected. But really, technology is ALWAYS changing. If you aren't continuously developing your skills, eventually you'll be out of a job. If the next major transition in networking is repeatable, scripted/reciped/etc. configurations, then you'd better learn how to write those scripts/recipes.

      Delete
  2. Email, terrible idea, think of all those unemployed postmen and women. Better to consider how you might evolve your skillset than make such arguments and complain about progress.

    Funnily enough my very first blog post was about this very subject: http://packetpushers.net/the-man-in-the-white-suit/

    ReplyDelete
  3. We use maintenance windows because not everything goes as planned. Not because we don't trust manual processes. Manual processes do cause a lot of issues. But no matter how many times we test something in the lab, they do occasionally fail. We do them in maintenance windows to reduce the risk that a failure would affect the business. Even if we had a hundred successful windows in a row, changes to business systems will happen off-hours. Automated or not. This isn't old-thinking, it's reality.

    ReplyDelete
    Replies
    1. And how come the virtualization admins can create a new VM, new port group, or a new LUN or shared file system at any time?

      Delete
    2. I don't think all of those are equal and a well managed shop won't have admins doing things at any time.

      A new VM could drive a host over the edge resource wise, a new LUN could be placed in the wrong place overloading a SAN and causing performance issues for a critical application.

      Delete
  4. I can't believe ppl are worried about job security and threatened by automation? I for one can think of more creative ways to flex my brain cells than configuring access ports. VLANs etc.. I want to be freed up from the error prone CLI mashing to actually get a chance to learn new skills like Openstack and SDN

    ReplyDelete
  5. Nice and very illustrative article.

    And agrede with Amanda,,, I feel that exciting challenges and opportunities are coming, and if we think that all we can contribute as NW engineers is configure ports and VLANs, then we have a real problem... I would expect that we can provide much more value than simply memorise a set of commands and then blindly repeat them.. We understand how complex computer systems communicate and the challenges they present... And is up to us to keep learning and evolving,,, that is the fun part, isn't it?

    I don't know... but I think that to reduce the SDN wave to a debate around NW engineers versus programmer profiles (or OMG! I'm going to be replaced by a script!) is a bit simplistic and underestimate ourselves.

    ReplyDelete
  6. Some of the points you make, particularly about transactions reminds me of the Clark Wilson Integrity model.

    http://theory.stanford.edu/~ninghui/courses/Fall03/papers/clark_wilson.pdf

    ReplyDelete
  7. I definitely found this idea interesting...I have started really liking the idea of network virtualization for allowing increased network automation, but it obviously has limits (what about the enterprise WAN, service providers, underlying physical devices in the DC).

    The above would make a commit/rollback mechanism more important in the network (just as in a database).

    You could also probably still leverage virtual environments for some of your testing i.e. test changes in a part virtual/part physical environment.

    Also we need standardization of some of these programming interfaces so that we can write code that can work relatively easily across multiple vendors (maybe OVSDB?...need to look into this more).

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Sidebar