Declarative and Procedural Programming (and How I Got It all Wrong)
During a recent NetOps-focused discussion trying to figure out where Puppet/Chef/Ansible/… make sense in the brave new SDN-focused networking world I made this analogy: “Puppet manifest is like Prolog, router configuration is like Java or C++.” It’s a nice sound bite. It’s also totally wrong.
Declarative versus Procedural Programming
In a nutshell, declarative programming languages allow you to tell what needs to be done, not how to do it (which is what procedural programming languages are all about).
Unfortunately nothing good just happens to happen (at least not in IT world, or it might have to do with the second law of thermodynamics), you need something that will make the transition (or find a path) from where we are now to the final state (the what). The something might be a language interpreter (in Prolog’s case performing an exhaustive search of the solution tree to figure out if it can satisfy the requirements specified in the declarative program) or a Puppet agent that:
- Compares the current state of the system (example: web server) with the desired state (Puppet manifest);
- Figures out the differences;
- Applies changes (modifying configurations, creating files, executing commands…) that have to be made to transition the system from the current state to the desired state.
It’s obvious router configurations aren’t procedural programs – they tell a device (or software) what needs to be done not how that behavior should be implemented. In most cases you cannot influence the implementation details; EEM applets are an obvious exception, but the only reason they reside in router configuration is the lack of decent text editing tools on network devices convenience of manipulating router configuration as compared to anything else.
So What’s the Difference?
The real difference between Puppet manifests and router configurations is the level of abstraction. Puppet manifests should focus on resources (example: VLANs) and desired state of resources (example: VLAN 10 present on port GigabitEthernet0/1) not on the implementation details.
Did I say implementation details? Isn’t that procedural programming? No, we’re still in the realm of declarative programming (we never tell the switches how to implement VLANs, do we?), but working at a lower level of abstraction and dealing with how individual devices (or vendors) expect things to be declared.
Confusing? Sure it is, but don’t worry. It’s no more confusing than other things we have to deal with. You just need a bit of practice… and don’t forget to focus on principles, not implementation details.
I wonder if this makes sense to you as the way to decide what is procedural and what is declarative.
Consider the early history of programming languages for the first three generations: machine code, assembler, 3rd generation languages (COBOL, FORTRAN, C, Basic, etc.). In each of these generations, the earlier generation is hidden by the abstractions of the next higher layer. The higher layer is the "declarative" language for the earlier "procedural" one. So the terms are relative to where you are standing in the hierarchy.
Or, said differently, "What" is handled at the current layer and its "How" is handled by the layer underneath it and so on.
I offer this model since it underscores what the value of higher levels of abstraction are, why higher levels don't obsolete lower levels and to clarify that procedural and declarative are meaningless until you state what level of the abstraction stack you are standing on. It also suggests that additional abstraction layers can be added, and are more valuable the less changes to the underlying levels of abstraction they require.
Consider a router CLI; it's declarative when you use it to configure a device. But it's procedural if you have a GUI on top in which a single mouse click "Add VLAN to Fabric" causes a series of CLI commands to be executed on all the switches and ports involved in order to accomplish the "How".
Perhaps the above is nonsense, but it's how I explain the difference.
Let me illustrate this by contrasting configuration in Linux (which is imperative) with configuration in the CLI of one of the major router vendors (which is imperative).
I will use a simple example - configuring an IP address on an interface.
In Linux you configure the IP address as follows:
$ sudo ifconfig eth0 10.10.10.1 netmask 255.255.255.0 up
Then you can see that the interface has the desired IP address as follows:
$ ifconfig eth0
eth0 Link encap:Ethernet HWaddr 08:00:27:21:b6:ee
inet addr:10.10.10.1 Bcast:10.10.10.255 Mask:255.255.255.0
...
In the router CLI you configure the IP address as follows:
interfaces {
ge-0/0/0 {
unit 0 {
family inet {
address 10.10.10.1/24;
}
}
}
}
And you can see the that the interface actually has the desired IP address by looking at the operational state:
> show interfaces ge-0/0/0
Physical interface: ge-0/0/0, Enabled, Physical link is Up
Logical interface ge-0/0/0.0 (Index 65541) (SNMP ifIndex 635)
Protocol inet, MTU: 1500
Destination: 10.10.10/24, Local: 10.10.10.1, Broadcast: 10.10.10.255
...
The router CLI syntax is clearly different from the Linux syntax, but other than that, there doesn't appear to be a significant difference.
Now let's make the example a little bit more interesting. Let's attempt to configure an IP address for an interface which is not yet present on the device, i.e. an interface on an interface card which has not yet been inserted in the router or server.
In Linux, if you attempt to do this, the command will fail. This is because the command is imperative. You tell Linux to do something right how. Since it cannot be done right now, the command fails.
$ sudo ifconfig eth1 20.20.20.1 netmask 255.255.255.0 up
SIOCSIFADDR: No such device
eth1: ERROR while getting interface flags: No such device
...
In the router CLI, if you attempt to commit the configuration it will succeed. This is because the configuration statement is imperative. You are telling the router that the desired state is for a particular interface to have a particular IP address. Even though there is no interface card in slot 1, the configuration for an interface in slot 1 is allowed to exist:
interfaces {
ge-1/0/0 {
unit 0 {
family inet {
address 20.20.20.1/24;
}
}
}
}
But if you use a different show command to look at the operational state, you will see that the interface is not there.
> show interfaces ge-1/0/0
error: device ge-1/0/0 not found
The router operating system is constantly trying to make the actual state (the operational state) equal to the desired state (the administrative state). All those attempts fail until you actually insert the interface card into the router. Then, suddenly and automatically without any manual intervention, the interface will appear in the operational state with the desired IP address.
(continued in next response)
This router CLI implementation also supports some interesting features allowing you to implement the higher level of abstraction which you mentioned in your post. It supports the concept of "apply macros" (link) which allows the end-user to introduce configuration macros at a higher level of abstraction in the router configuration which get mapped to the more detailed configuration statements using a mapping script.
It also support scripts which can enforce consistency rules designed by the end-user. For example, it could enforce that OSPF must be enabled on every interface and fail the commit if OSPF is missing on an interface.
Finally, the implementation of this particular CLI uses Yang data models to describe the structure of the configuration tree. As a result of that implementation decission, everything that can be configured by a human using human-readable CLI can also be configured programmatically using Netconf as a machine-to-machine protocol. There is a Python library available to make this easy to use (look for ezpy on Github).
The point of this response is to point out that you can actually do a lot more with the CLI of existing routers than most people realize, and that at least some implementation are already imperative, already support higher levels of abstraction, and already provide machine-to-machine interfaces and Pythin scripts for automation.
Thank you for these examples. It strikes me that one reason for netconf being developed was in reaction to the complexity in configuring network devices to achieve a "correct" configuration state. Or, said differently, proven software engineering tools (data models, configuration management tools, reusable and tested modules) reduce complexity and increase odds of achieving"correctness".
I think another byproduct of netconf is the ability encode the knowledge of skilled network designers (a growingly scarce resouce) so it can be resued by less skilled administrators. This is a common by-product of an abstraction hierarchy that we find everywhere, such as consumer products to cite one example.