Why is RESTful API better than SNMP?
Brian Christopher Raaen asked a great question in a comment to my OpenStack/Quantum SDN-Based Virtual Networks post:
Other than some syntax difference what do these new HTTP-based APIs add that SNMP couldn’t already do?
Short answer: In theory nothing, apart from familiarity and convenient programming libraries. In practice, there’s a huge gap between theory and practice. See “Hands-on experience” at the bottom of the article.
What is a RESTful service
If you’re not familiar with the concept of Representation State Transfer (REST) you might want to read this Wikipedia entry that describes REST as an architectural solution, not a specific protocol or data format.
The most interesting part of the article is the description of the REST constraints. A RESTful service should be a stateless client-server service – clearly SNMP fits that role well. It’s harder to shoehorn SNMP into other constraints (cacheable layered system), but these constraints aren’t that important in the network programming space.
Moving forward to the guiding principles of a REST interface, the differences with SNMP become more pronounced. SNMP requires an external schema (MIB) to describe the data model, whereas a REST interface should provide self-descriptive messages.
Finally, while you could (in principle) run a RESTful service over an RFC 1149-based infrastructure, most real-life implementations run over HTTP(S), reuse existing Web services authentication/authorization mechanisms (HTTP authentication or cookies), and use JSON or XML as the data transfer format.
Why is REST better than SNMP?
Imagine you’re looking for someone to write detailed technical documentation for your new SDN gizmo. Would it be simpler to find someone fluent in English or in Elbonian?
The situation is no different in the programming world – anyone can write a client script that uses HTTP(S) to send form data to a web server and process JSON response (XML-based requests and responses might already represent significant mental challenges to some youngsters).
HTTP(S) client is usually included with modern programming languages and available (at least as a library) in every single language I’ve tried to use in the last decade (before you ask: there was no HTTP client in Turbo Pascal, PL/I or Fortran IV when I was still using them), and a lot more programmers were exposed to HTTP than to SNMP, some of them probably never realizing where the HTTP part of the XMLHttpRequest object comes from.
Amazingly, there’s an SNMP library for PHP and Ruby … or you could use Erlang (although I still can’t figure out whether that’s a joke as the manual was published on April 1st).
The situation is no different on the server side. You can implement a RESTful server in any programming language using any open-source web server, from standalone PERL scripts and lightweight options to full-blown web servers. SNMPv3 server? Not so easy.
Summary
I can’t see a major functional difference between a RESTful service and SNMP (but see below). Also, I can’t see where SNMP would be better than a RESTful interface now that we’re past the days of Z80 processors being used in networking devices.
BTW, don’t tell me most SNMP MIBs are read-only. I’m well aware of that – but that’s not a limitation of the SNMP protocol, but of its implementations in specific network devices. Wellfleet was more than happy to have their routers fully configurable through SNMP … unfortunately that was the only configuration mechanism they offered, and their SNMP-based CLI was a royal pain.
However, regardless of what our opinions might be, REST is clearly a mainstream solution and SNMP is limited to a very specific field. Which one do you think will be used more?
Hands-On Experience
At the time this blog post was published, John Gruber wrote a great reply to the original question:
In scale, HTTP is quite a bit better than SNMP. RESTful APIs use HTTP GETs to retrieve data. SNMP is used extensively to retrieve stats data from devices. When the number of tenants retrieving stats from an infrastructure has to scale to the public, SNMP agents on various devices will indeed stress their control planes. Most embedded system control planes don’t have an excess of CPU cycles to burn as it is. Nor do they have the intelligence for rate limiting or caching of management requests in scale.
With HTTP we have a rich and well proven delivery and caching mechanism which can be used to impose appropriate limits simply by serving requests out of CHEAP and available application level RAM caches. HTTP delivery and rewrite proxies are available at a MUCH lower cost and point of entry than similar mechanism which use SNMP.
Even for validating provisioning requests, using HTTP POST or PUT, verse SNMP sets, opens up a world of scripting and coding that SNMP doesn’t support readily. Everyone can setup an HTTP rewrite and cache engine in half a dozen scripting languages. How many of us can say the same for SNMP?
While SNMP is wonderful, it’s not as accessible or as cheap to get working well as HTTP in our day and age. That’s why we are all moving towards HTTP as an application level protocol and web based data structures, like JSON. Ask yourself how many of us thought HTTP would be the world’s most popular transport for video 5 years ago? It is. It may not be as optimal for the job as RTP, but it has won the day. Viva La Web..
Tristan Colgate-McFarlane added his thoughts in a comment copied into the main blog post for your convenience:
HTTP/REST is certainly easier in some respects, but SNMP does have quite a few advantages. It supports types, MIBs are an excellent source of documentation for an API and those APIs tend to be considerably more stable than their RESTful cousins (the RabbitMQ management plugin API has changed in every single version I’ve deployed and broken nagios checks every time). Many of these APIs are even standardised (and some vendors even stick to them sometimes). There are some missing things, like properly standardised floats and doubles.
Not to mention traps, a decent (if loathed) security model in v3, no TCP 3 way handshake (unless you want it). Stuff like EVENT-MIB and EXPRESSION-MIB give you standard server-side functionality. net-snmp does a pretty good job of doing most of the really hard stuff for you.
The real problem for SNMP is that it is something else to learn and doesn’t work well if you stick to the bare minimum. Doing tables properly is hard (that’s trivial in a XML or JSON based REST API). People don’t even think about using things like contexts. MIBs are also scarily close to actually writing documentation, which is never going to go down well with the majority of developers. It hasn’t helped that most examples of SNMP usage try and bypass MIBs completely and use numeric oids for everything which makes examples practically impossible to read and gives SNMP a scary, super complicated air about it. You can do without MIBs, but that doesn’t mean you should, you can always replace the textual values with the OIDs once you’ve worked everything out.
I think SNMP’s time may have come, but it has merits that we are losing with HTTP REST APIs, just not ones that people care about that much. How much they should care is open for debate.
Finally, in September 2020 Rich Smith described his experience with SNMP nightmares:
I know this article is very old by now, but as someone who spent many years working with SNMP, I strongly disagree with the conclusion. SNMP has one (possible) advantage over REST, which is that it is connectionless. After that, it is a horror show.
The biggest issue is probably the amount of space that OIDs consume in limited-size packets. GetBulk did not end up implementing any OID compression, and so when walking a table of counters, gauges, or states, for example, almost the entire, precious 1500 byte packet capacity is consumed transporting OIDs. In the real world, packet fragmentation and reassembly does not work well, and so you have to carefully craft your request packets to avoid size overruns in responses.
The only traversal semantic is SNMP is GetNext/GetBulk. There is no filtering, that I am aware of, and scoping is only available by specifying more of the OID prefix, which is limiting.
Table walk semantics are difficult to implement correctly for sparse tables. Even if you implement that correctly, you have to be careful which other columns your GetBulk runs over into at the end of the table. We had a case where the last row of a table could never be fetched due to running over into an unrelated table with a too-large OCTET STRING. We had to create a custom table-walk algorithm for that specific table.
Because many agents are single-threaded, you have to implement throttling on the management side to avoid spurious timeouts and retries. (Without throttling, requests queue up in the agent, including retries, and everything times out.)
CRUD semantics for tables are difficult to implement correctly. At my last job, we almost got sued by a customer because one out of a sequence of SetRequest packets failed, and left a wide-open WLAN on their network. Since SetRequest is not idempotent, you cannot even retry those.
There are other parts of the protocol I could pick on, such as community strings for authentication, or OID index parsing, or the difficult-to-use InetAddress/InetAddressType data types. There is a lot that did not age well.
Another commenter stated that SNMP is a terrible protocol that needs to die. I don’t know if I’d go that far, but for managing a device of any complexity, it is so much better to implement a REST API. You have reliability and the ability to stream large amounts of data, if needed. Scoping and filtering are available via URLs and query parameters. JSON encoding is not even much more verbose than all of the SNMP OIDs that come back in GetBulk. When you ask for a table of data, you don’t get unrelated tables. You don’t have to implement “sparse walk”.
REST APIs also allow for various authentication schemes that HTTP supports. TLS is supported simply.
There are dozens of open source libraries and frameworks for HTTPS, REST, and JSON.
REST is just superior. Anyone who says “SNMP can do anything that REST does” has, I would wager, never spent much time maintaining a sophisticated SNMP-based management application. I have done that, and I guarantee that the statement that the two protocols are equivalent is naive.
Also, you might enjoy dozens of comments that accumulated since the original blog post was published in 2012.
Not to mention traps, a decent (if loathed) security model in v3, no TCP 3 way handshake (unless you want it). Stuff like EVENT-MIB and EXPRESSION-MIB give you standard server-side functionality. net-snmp does a pretty good job of doing most of the really hard stuff for you.
The real problem for SNMP is that it is something else to learn and doesn't work well if you stick to the bare minimum. Doing tables properly is hard (that's trivial in a XML or JSON based REST API). People don't even think about using things like contexts. MIBs are also scarily close to actually writing documentation, which is never going to go down well with the majority of developers. It hasn't helped that most examples of SNMP usage try and bypass MIBs completely and use numeric oids for everything which makes examples practically impossible to read and gives SNMP a scary, super complicated air about it. You can do without MIBs, but that doesn't mean you should, you can always replace the textual values with the OIDs once you've worked everything out.
I think SNMP's time may have come, but it has merits that we are losing with HTTP REST APIs, just not ones that people care about that much. How much they should care is open for debate.
SNMP GET and HTTP GET doesn't consume the same amount
On slow links that remain in the wild, it would be a shame that supervision traffic consumes all bandwidth, especially when doing "GET BULK".
both SNMP and HTTP are application layer protocols,
can you please tell me which one takes more bandwidth and how ?
The bottom line is that we wanted a tool to allow us to run either efficient like SNMP or a bit more convenient like REST, so we developed a pretty neat control protocol and released it on GitHub. I don't know if it would replace SNMP in anyone's system, but it might be worth a look see.
https://github.com/democosm/hc-cpp
Some extra detail is here:
http://www.democosm.com
As Tristan points out, SNMP tables are utterly brain-dead, just think of all the hassles with mutable ifIndexes. A text-based protocol is inherently more flexible than something relying on ASN.1 OIDs The original sin of SNMP is its use of OSI technology. Just think of how many security advisories have been caused by ASN.1 bugs over the years.
SNMP was suppose to create a unified cross-platform device interaction framework, however it did not. The same thing is being attempted and proposed with these new solutions. I really don't see them fixing it, since it is not the technology that is the limiting factor but rather the way people implement it. I think imaging a grand unified cross platform/vendor interaction framework is really a pipe dream.
Thanks for the discussion Ivan, I'm really enjoying it. I really enjoy getting people to think about things so we can improve ourselves and each other.
Reading and writing awkward two column tables that are never indexed by the value you need is not an API.
It is fine for counters, but everything else is needlessly complicated.
This is probably the simplest example of how broken SNMP is as an API: http://www.cisco.com/en/US/tech/tk648/tk362/technologies_tech_note09186a00801c9199.shtml
You need a 6 step process to reproduce "show mac-address-table address 0000.0c07.ac08"
I have a python module that is few hundred lines of code that parses telnet/ssh output for switches. It lets me do things like write a program that enables dhcp snooping automatically based on the current vlan configuration and network topology.
I have another program that you can run on a group of switches and give it 1 or more old vlans and 1 or more new vlans. It will automatically add the new vlans to any trunk ports that have the old vlans.
These are the kinds of things a real API would help with.
Your way means you need to learn the command for every type of system, work out how to access the box, write a parser for the command output, deal with changes from CLI revisions etc.
SNMP works for big system management because it works the same way every time.
IETF has been dragging its feet for a decade, but finally we're getting standard data models for NETCONF/RESTCONF and they're slowly getting implemented. It will take another few years to get anywhere close to where MIBs are today... assuming you can get the information you need from the standard MIBs. Yes, you can get MAC table. No, you can't get optical interface signal levels.
And SNMP isn't a Transport: that's UDP or TCP. SNMP describes a number of Protocol Data Units (PDU's) or put more simply - message formats.
SNMP has verbs such as GET, GETNEXT, SET - in the same way that HTTP has POST, GET, PUT, DELETE etc.
And whilst using HTTP GET to find out the IP address of an interface such as:
http://some_device/restconf/api/config/native/interface/GigabitEthernet/2/ip/address
certainly looks more friendly that the SNMP equivalent there is still a huge amount of codification effort to make it work reliably across the industry. Which is why it has been 10 years - and not looking "standard" anytime soon.
I'm not banging the drum for SNMP here; it is not easy to understand or use - partly because of the simplicity of the protocol. All I'm saying is that whether you use REST, WBEM, WMI, or SNMP there is no magic bullet to getting an agreed standard "language" to allow you to gather the data you want reliably across the 1000's of different hardware and software platforms across the IT industry.
I had just started putting together an article for Packet Pushers concerning a similar topic. Below is the gist of post:
From what I understand, Cisco will be releasing their OnePK SDN offering as a language-specific API, starting with JAVA, with no clear roadmap afterwards. Instead, Cisco should initially release OnePK as a RESTful API. A Cross-Platform RESTful API would allow programmers and systems-scripters alike to adopt OnePK as their SDN platform from Day One.
Examples:
(1) Ethan could write Perl scripts to manage TCAM or routing tables (GET, PUT, POST, DELETE, etc)
(2) You could write scalable webapps for, say, network monitoring, in Ruby, PHP, etc.
(3) Enterprises with an internal development team would be able to write .NET applications for integrated inventory and network provisioning.
Providing OnePK as a cross-platform RESTful API would allow immediate adoption by a considerably broader audience, many of whom influence the decision of which SDN strategy to adopt in their environment.
-Mike
I'm not sure how useless a Java-specific API is, since Java does have a dis-proportioned amount of the enterprise stack. On the whole I agree with you that they could've done better by maybe releasing a binary format like Avro, Thrift, Protocol Buffers, or maybe even Etch, since they did create it and it is supposed to be serialization for "consuming network services"
Also regarding a comment about bandwidth requirements vs. snmp; if you're doing a lot of data fetching, you may be able to leverage http's support of compressed content-encoding, making this much less of an issue.
"How I Explained REST to My Wife"
http://tomayko.com/writings/rest-to-my-wife
-Mike
Short answer: Do you understand FCAPS
I find application/host programmers do not share the same experience in fielded products
that as networking/embedded programmers.
So if all you have is a hammer clearly its the right tool to use.
HTTP does not enforce this coupling of the transport and the message content. Whilst it is possible to request a particular MIME type encoding (i.e. text/html), the format of the response is not defined by the HTTP protocol specification. Of course, the client and server can use a MIME-type which has the ability to use schema to describe the messages (such as XML) - but then you're starting to look more SOAPy and less RESTy.
I don't know if there are existing standards that provide schemas for defining the messages content - if there aren't any then the future will be painful for anyone who wants to interact with multiple vendors each of whom is using their own schema.
To avoid this, I would like to propose the intriguing possibility of adding a new MIME type of text/snmp - allowing the HTTP REST'ful approach to be used with the schema and data types of SNMP!
For example a HTTP GET with the to the URL:
http://switch1/get_status/oid/1.3.6.1.2.1.1.3
would return the ASN1.1 representation of the switches system uptime (or perhaps a JSON representation of the same).
Perhaps this could be made part of the FCoTR alliance?
PS. After I had this novel idea (to me anyhow), I searched and found this RFC draft: http://tools.ietf.org/id/draft-mellquist-web-sys-00.txt - which is similar in scope, although they just POST the SNMP PDU in the HTTP body don't use the OID in the path of the URL.
What do you gain?
So sure, using REST to gather data would work just fine - relatively inefficient, but fine. But you'd need the entire industry to agree on just how to request the optical signal levels, or mac table from any given device.
I'm not holding my breath on that one.
I can say with all seriousness: SNMP is a terrible protocol, and should die a horrible death.
SNMP 1 and 2 are very similar, although SNMP 2 has three distinct versions: 2a, 2b, 2c. Only 2c is really used. The biggest difference is that it enabled 64-bit counters. Both 1 and 2x are unencrypted.
SNMP 3 changed a lot, including adding very crappy encryption (a simple pre-shared key). It was almost a decade before vendors started to implement SNMP v3 in any significant way.
MIB files are a huge pain in the ass to parse, especially with your eyeball. The tools available are equally as crappy.
RESTful APIs allow a much easier way to get access to data and do configuration. XML is a shitload easier to parse than a MIB file. And REST allows PKI (SSL/TLS and certificates and such) which offer much better (potential) security.
Like modem init strings, OIDs haunt my dreams at night. Both are things I hope I never run into again.
SNMP: Simple Network Management Protocol. The biggest lying acronym in IT :)
REST is certainly a hell of a lot more powerful than SNMP. But for the purpose of extracting stats out of boxes, SNMP is amazing because it is cross vendor compatible. I can pull stats from Juniper and Cisco and you name it routers with the same command / set of commands. I don't have to write a separate adapter for every little gizmo in the world.
I know this article is very old by now, but as someone who spent many years working with SNMP, I strongly disagree with the conclusion. SNMP has one (possible) advantage over REST, which is that it is connectionless. After that, it is a horror show.
The biggest issue is probably the amount of space that OIDs consume in limited-size packets. GetBulk did not end up implementing any OID compression, and so when walking a table of counters, gauges, or states, for example, almost the entire, precious 1500 byte packet capacity is consumed transporting OIDs. In the real world, packet fragmentation and reassembly does not work well, and so you have to carefully craft your request packets to avoid size overruns in responses.
The only traversal semantic is SNMP is GetNext/GetBulk. There is no filtering, that I am aware of, and scoping is only available by specifying more of the OID prefix, which is limiting.
Table walk semantics are difficult to implement correctly for sparse tables. Even if you implement that correctly, you have to be careful which other columns your GetBulk runs over into at the end of the table. We had a case where the last row of a table could never be fetched due to running over into an unrelated table with a too-large OCTET STRING. We had to create a custom table-walk algorithm for that specific table.
Because many agents are single-threaded, you have to implement throttling on the management side to avoid spurious timeouts and retries. (Without throttling, requests queue up in the agent, including retries, and everything times out.)
CRUD semantics for tables are difficult to implement correctly. At my last job, we almost got sued by a customer because one out of a sequence of SetRequest packets failed, and left a wide-open WLAN on their network. Since SetRequest is not idempotent, you cannot even retry those.
There are other parts of the protocol I could pick on, such as community strings for authentication, or OID index parsing, or the difficult-to-use InetAddress/InetAddressType data types. There is a lot that did not age well.
Another commenter stated that SNMP is a terrible protocol that needs to die. I don't know if I'd go that far, but for managing a device of any complexity, it is so much better to implement a REST API. You have reliability and the ability to stream large amounts of data, if needed. Scoping and filtering are available via URLs and query parameters. JSON encoding is not even much more verbose than all of the SNMP OIDs that come back in GetBulk. When you ask for a table of data, you don't get unrelated tables. You don't have to implement "sparse walk".
REST APIs also allow for various authentication schemes that HTTP supports. TLS is supported simply.
There are dozens of open source libraries and frameworks for HTTPS, REST, and JSON.
REST is just superior. Anyone who says "SNMP can do anything that REST does" has, I would wager, never spent much time maintaining a sophisticated SNMP-based management application. I have done that, and I guarantee that the statement that the two protocols are equivalent is naive.