Why is RESTful API better than SNMP?

Brian Christopher Raaen asked a great question in a comment to my OpenStack/Quantum SDN-Based Virtual Networks post:

Other than some syntax difference what do these new HTTP-based APIs add that SNMP couldn’t already do?

Short answer: In theory nothing, apart from familiarity and convenient programming libraries. In practice, there’s a huge gap between theory and practice. See “Hands-on experience” at the bottom of the article.

This post was edited and updated based on readers’ comments in September 2020.

What is a RESTful service

If you’re not familiar with the concept of Representation State Transfer (REST) you might want to read this Wikipedia entry that describes REST as an architectural solution, not a specific protocol or data format.

The most interesting part of the article is the description of the REST constraints. A RESTful service should be a stateless client-server service – clearly SNMP fits that role well. It’s harder to shoehorn SNMP into other constraints (cacheable layered system), but these constraints aren’t that important in the network programming space.

Moving forward to the guiding principles of a REST interface, the differences with SNMP become more pronounced. SNMP requires an external schema (MIB) to describe the data model, whereas a REST interface should provide self-descriptive messages.

Finally, while you could (in principle) run a RESTful service over an RFC 1149-based infrastructure, most real-life implementations run over HTTP(S), reuse existing Web services authentication/authorization mechanisms (HTTP authentication or cookies), and use JSON or XML as the data transfer format.

Why is REST better than SNMP?

Imagine you’re looking for someone to write detailed technical documentation for your new SDN gizmo. Would it be simpler to find someone fluent in English or in Elbonian?

The situation is no different in the programming world – anyone can write a client script that uses HTTP(S) to send form data to a web server and process JSON response (XML-based requests and responses might already represent significant mental challenges to some youngsters).

HTTP(S) client is usually included with modern programming languages and available (at least as a library) in every single language I’ve tried to use in the last decade (before you ask: there was no HTTP client in Turbo Pascal, PL/I or Fortran IV when I was still using them), and a lot more programmers were exposed to HTTP than to SNMP, some of them probably never realizing where the HTTP part of the XMLHttpRequest object comes from.

Amazingly, there’s an SNMP library for PHP and Ruby … or you could use Erlang (although I still can’t figure out whether that’s a joke as the manual was published on April 1st).

The situation is no different on the server side. You can implement a RESTful server in any programming language using any open-source web server, from standalone PERL scripts and lightweight options to full-blown web servers. SNMPv3 server? Not so easy.

Summary

I can’t see a major functional difference between a RESTful service and SNMP (but see below). Also, I can’t see where SNMP would be better than a RESTful interface now that we’re past the days of Z80 processors being used in networking devices.

BTW, don’t tell me most SNMP MIBs are read-only. I’m well aware of that – but that’s not a limitation of the SNMP protocol, but of its implementations in specific network devices. Wellfleet was more than happy to have their routers fully configurable through SNMP … unfortunately that was the only configuration mechanism they offered, and their SNMP-based CLI was a royal pain.

However, regardless of what our opinions might be, REST is clearly a mainstream solution and SNMP is limited to a very specific field. Which one do you think will be used more?

Hands-On Experience

At the time this blog post was published, John Gruber wrote a great reply to the original question:


In scale, HTTP is quite a bit better than SNMP. RESTful APIs use HTTP GETs to retrieve data. SNMP is used extensively to retrieve stats data from devices. When the number of tenants retrieving stats from an infrastructure has to scale to the public, SNMP agents on various devices will indeed stress their control planes. Most embedded system control planes don’t have an excess of CPU cycles to burn as it is. Nor do they have the intelligence for rate limiting or caching of management requests in scale.

With HTTP we have a rich and well proven delivery and caching mechanism which can be used to impose appropriate limits simply by serving requests out of CHEAP and available application level RAM caches. HTTP delivery and rewrite proxies are available at a MUCH lower cost and point of entry than similar mechanism which use SNMP.

Even for validating provisioning requests, using HTTP POST or PUT, verse SNMP sets, opens up a world of scripting and coding that SNMP doesn’t support readily. Everyone can setup an HTTP rewrite and cache engine in half a dozen scripting languages. How many of us can say the same for SNMP?

While SNMP is wonderful, it’s not as accessible or as cheap to get working well as HTTP in our day and age. That’s why we are all moving towards HTTP as an application level protocol and web based data structures, like JSON. Ask yourself how many of us thought HTTP would be the world’s most popular transport for video 5 years ago? It is. It may not be as optimal for the job as RTP, but it has won the day. Viva La Web..


Tristan Colgate-McFarlane added his thoughts in a comment copied into the main blog post for your convenience:


HTTP/REST is certainly easier in some respects, but SNMP does have quite a few advantages. It supports types, MIBs are an excellent source of documentation for an API and those APIs tend to be considerably more stable than their RESTful cousins (the RabbitMQ management plugin API has changed in every single version I’ve deployed and broken nagios checks every time). Many of these APIs are even standardised (and some vendors even stick to them sometimes). There are some missing things, like properly standardised floats and doubles.

Not to mention traps, a decent (if loathed) security model in v3, no TCP 3 way handshake (unless you want it). Stuff like EVENT-MIB and EXPRESSION-MIB give you standard server-side functionality. net-snmp does a pretty good job of doing most of the really hard stuff for you.

The real problem for SNMP is that it is something else to learn and doesn’t work well if you stick to the bare minimum. Doing tables properly is hard (that’s trivial in a XML or JSON based REST API). People don’t even think about using things like contexts. MIBs are also scarily close to actually writing documentation, which is never going to go down well with the majority of developers. It hasn’t helped that most examples of SNMP usage try and bypass MIBs completely and use numeric oids for everything which makes examples practically impossible to read and gives SNMP a scary, super complicated air about it. You can do without MIBs, but that doesn’t mean you should, you can always replace the textual values with the OIDs once you’ve worked everything out.

I think SNMP’s time may have come, but it has merits that we are losing with HTTP REST APIs, just not ones that people care about that much. How much they should care is open for debate.


Finally, in September 2020 Rich Smith described his experience with SNMP nightmares:


I know this article is very old by now, but as someone who spent many years working with SNMP, I strongly disagree with the conclusion. SNMP has one (possible) advantage over REST, which is that it is connectionless. After that, it is a horror show.

The biggest issue is probably the amount of space that OIDs consume in limited-size packets. GetBulk did not end up implementing any OID compression, and so when walking a table of counters, gauges, or states, for example, almost the entire, precious 1500 byte packet capacity is consumed transporting OIDs. In the real world, packet fragmentation and reassembly does not work well, and so you have to carefully craft your request packets to avoid size overruns in responses.

The only traversal semantic is SNMP is GetNext/GetBulk. There is no filtering, that I am aware of, and scoping is only available by specifying more of the OID prefix, which is limiting.

Table walk semantics are difficult to implement correctly for sparse tables. Even if you implement that correctly, you have to be careful which other columns your GetBulk runs over into at the end of the table. We had a case where the last row of a table could never be fetched due to running over into an unrelated table with a too-large OCTET STRING. We had to create a custom table-walk algorithm for that specific table.

Because many agents are single-threaded, you have to implement throttling on the management side to avoid spurious timeouts and retries. (Without throttling, requests queue up in the agent, including retries, and everything times out.)

CRUD semantics for tables are difficult to implement correctly. At my last job, we almost got sued by a customer because one out of a sequence of SetRequest packets failed, and left a wide-open WLAN on their network. Since SetRequest is not idempotent, you cannot even retry those.

There are other parts of the protocol I could pick on, such as community strings for authentication, or OID index parsing, or the difficult-to-use InetAddress/InetAddressType data types. There is a lot that did not age well.

Another commenter stated that SNMP is a terrible protocol that needs to die. I don’t know if I’d go that far, but for managing a device of any complexity, it is so much better to implement a REST API. You have reliability and the ability to stream large amounts of data, if needed. Scoping and filtering are available via URLs and query parameters. JSON encoding is not even much more verbose than all of the SNMP OIDs that come back in GetBulk. When you ask for a table of data, you don’t get unrelated tables. You don’t have to implement “sparse walk”.

REST APIs also allow for various authentication schemes that HTTP supports. TLS is supported simply.

There are dozens of open source libraries and frameworks for HTTPS, REST, and JSON.

REST is just superior. Anyone who says “SNMP can do anything that REST does” has, I would wager, never spent much time maintaining a sophisticated SNMP-based management application. I have done that, and I guarantee that the statement that the two protocols are equivalent is naive.


Also, you might enjoy dozens of comments that accumulated since the original blog post was published in 2012.

32 comments:

  1. HTTP/REST is certainly easier in some respects, but SNMP does have quite a few advantages. It supports types, MIBs are an excellent source of documentation for an API and those APIs tend to be considerably more stable than their RESTful cousins (the RabbitMQ management plugin API has changed in every single version I've deployed and broken nagios checks every time). Many of these APIs are even standardised (and some vendors even stick to them sometimes). There are some missing things, like properly standardised floats and doubles.

    Not to mention traps, a decent (if loathed) security model in v3, no TCP 3 way handshake (unless you want it). Stuff like EVENT-MIB and EXPRESSION-MIB give you standard server-side functionality. net-snmp does a pretty good job of doing most of the really hard stuff for you.

    The real problem for SNMP is that it is something else to learn and doesn't work well if you stick to the bare minimum. Doing tables properly is hard (that's trivial in a XML or JSON based REST API). People don't even think about using things like contexts. MIBs are also scarily close to actually writing documentation, which is never going to go down well with the majority of developers. It hasn't helped that most examples of SNMP usage try and bypass MIBs completely and use numeric oids for everything which makes examples practically impossible to read and gives SNMP a scary, super complicated air about it. You can do without MIBs, but that doesn't mean you should, you can always replace the textual values with the OIDs once you've worked everything out.

    I think SNMP's time may have come, but it has merits that we are losing with HTTP REST APIs, just not ones that people care about that much. How much they should care is open for debate.

  2. Don't forget about bandwidth....
    SNMP GET and HTTP GET doesn't consume the same amount

    On slow links that remain in the wild, it would be a shame that supervision traffic consumes all bandwidth, especially when doing "GET BULK".
    Replies
    1. So does that mean, SNMP GET requires more bandwidth ?
      both SNMP and HTTP are application layer protocols,
      can you please tell me which one takes more bandwidth and how ?
    2. Er no. At its simplest a single SNMP get will only require 2 packets as it's UDP. HTTP is a TCP protocol and has a far greater overhead
    3. The company I work for is trying to move away from SNMP and toward a RESTful API (web GUI) for our embedded systems that we make. When they looked into it, the thinking was that SNMP can be a bitter a pill for people to swallow in the embedded Linux market. However, they also noted that tying our C++ code into the backend of a RESTful API would also be a pain.
      The bottom line is that we wanted a tool to allow us to run either efficient like SNMP or a bit more convenient like REST, so we developed a pretty neat control protocol and released it on GitHub. I don't know if it would replace SNMP in anyone's system, but it might be worth a look see.
      https://github.com/democosm/hc-cpp
      Some extra detail is here:
      http://www.democosm.com
  3. Python has excellent SNMP support. I've implemented custom agents in a couple of pages of code, and most of the effort lay in MIB design to implement GETNEXT

    As Tristan points out, SNMP tables are utterly brain-dead, just think of all the hassles with mutable ifIndexes. A text-based protocol is inherently more flexible than something relying on ASN.1 OIDs The original sin of SNMP is its use of OSI technology. Just think of how many security advisories have been caused by ASN.1 bugs over the years.
    Replies
    1. Well, I wouldn't say that ASN.1 bugs were caused by it being an OSI technology ;) Judged by the "quality" of XML parser in Cisco IOS, being a W3C or IETF technology doesn't help much.
  4. While I think SNMP will still be with us for some time to come, especially for graphing trends and alerting with traps, I see a lot of vendors moving to the newer http based methods. The bigger problem with snmp was how vendors implemented convoluted MIBs that were not documented well and were difficult to understand. Also every vendor (as well as internal group) had their own way of doing things that was different. I really don't see that same fundamental problem changing with the xml based technologies since everyone will have their own schema that works differently.

    SNMP was suppose to create a unified cross-platform device interaction framework, however it did not. The same thing is being attempted and proposed with these new solutions. I really don't see them fixing it, since it is not the technology that is the limiting factor but rather the way people implement it. I think imaging a grand unified cross platform/vendor interaction framework is really a pipe dream.

    Thanks for the discussion Ivan, I'm really enjoying it. I really enjoy getting people to think about things so we can improve ourselves and each other.
  5. SNMP needs to die.

    Reading and writing awkward two column tables that are never indexed by the value you need is not an API.

    It is fine for counters, but everything else is needlessly complicated.

    This is probably the simplest example of how broken SNMP is as an API: http://www.cisco.com/en/US/tech/tk648/tk362/technologies_tech_note09186a00801c9199.shtml

    You need a 6 step process to reproduce "show mac-address-table address 0000.0c07.ac08"

    I have a python module that is few hundred lines of code that parses telnet/ssh output for switches. It lets me do things like write a program that enables dhcp snooping automatically based on the current vlan configuration and network topology.

    I have another program that you can run on a group of switches and give it 1 or more old vlans and 1 or more new vlans. It will automatically add the new vlans to any trunk ports that have the old vlans.

    These are the kinds of things a real API would help with.
    Replies
    1. Not sure why google changed my name to "Unknown" but I think I fixed it :-)
    2. I think you're missing the point of SNMP. You might need a few steps to reproduce the "show mac ..." with SNMP, but once you've done that it will work for every device in the world that supports that mib.

      Your way means you need to learn the command for every type of system, work out how to access the box, write a parser for the command output, deal with changes from CLI revisions etc.

      SNMP works for big system management because it works the same way every time.
    3. @Dave: I think you're missing the difference between "SNMP as transport" and "standard data models - i.e. MIBs".

      IETF has been dragging its feet for a decade, but finally we're getting standard data models for NETCONF/RESTCONF and they're slowly getting implemented. It will take another few years to get anywhere close to where MIBs are today... assuming you can get the information you need from the standard MIBs. Yes, you can get MAC table. No, you can't get optical interface signal levels.
    4. @Ivan - not getting a particular value for some system property is only because there is no MIB - not because of any inherent limitation in SNMP.

      And SNMP isn't a Transport: that's UDP or TCP. SNMP describes a number of Protocol Data Units (PDU's) or put more simply - message formats.

      SNMP has verbs such as GET, GETNEXT, SET - in the same way that HTTP has POST, GET, PUT, DELETE etc.

      And whilst using HTTP GET to find out the IP address of an interface such as:

      http://some_device/restconf/api/config/native/interface/GigabitEthernet/2/ip/address

      certainly looks more friendly that the SNMP equivalent there is still a huge amount of codification effort to make it work reliably across the industry. Which is why it has been 10 years - and not looking "standard" anytime soon.

      I'm not banging the drum for SNMP here; it is not easy to understand or use - partly because of the simplicity of the protocol. All I'm saying is that whether you use REST, WBEM, WMI, or SNMP there is no magic bullet to getting an agreed standard "language" to allow you to gather the data you want reliably across the 1000's of different hardware and software platforms across the IT industry.
    5. And now we're in perfect sync ;)
  6. Hi Ivan -
    I had just started putting together an article for Packet Pushers concerning a similar topic. Below is the gist of post:

    From what I understand, Cisco will be releasing their OnePK SDN offering as a language-specific API, starting with JAVA, with no clear roadmap afterwards. Instead, Cisco should initially release OnePK as a RESTful API. A Cross-Platform RESTful API would allow programmers and systems-scripters alike to adopt OnePK as their SDN platform from Day One.

    Examples:
    (1) Ethan could write Perl scripts to manage TCAM or routing tables (GET, PUT, POST, DELETE, etc)
    (2) You could write scalable webapps for, say, network monitoring, in Ruby, PHP, etc.
    (3) Enterprises with an internal development team would be able to write .NET applications for integrated inventory and network provisioning.

    Providing OnePK as a cross-platform RESTful API would allow immediate adoption by a considerably broader audience, many of whom influence the decision of which SDN strategy to adopt in their environment.

    -Mike
    Replies
    1. Totally agree with you. Java-specific API is useless (at least for me and a few others).
    2. Eh-late to the game on this one, but it is an interesting topic.

      I'm not sure how useless a Java-specific API is, since Java does have a dis-proportioned amount of the enterprise stack. On the whole I agree with you that they could've done better by maybe releasing a binary format like Avro, Thrift, Protocol Buffers, or maybe even Etch, since they did create it and it is supposed to be serialization for "consuming network services"
  7. Another benefit of restful could be integrated monitoring client. Since you're already doing HTTP, it shouldn't be hard to serve a set of static (server-wise) webpages with javascript that fetch data and show it in real time. All that just by browsing to (mgmt) ip of device.

    Also regarding a comment about bandwidth requirements vs. snmp; if you're doing a lot of data fetching, you may be able to leverage http's support of compressed content-encoding, making this much less of an issue.
  8. There's a cute descritpion of REST at:

    "How I Explained REST to My Wife"
    http://tomayko.com/writings/rest-to-my-wife

    -Mike
  9. >Short answer: nothing, apart from familiarity and convenient programming libraries.

    Short answer: Do you understand FCAPS
    I find application/host programmers do not share the same experience in fielded products
    that as networking/embedded programmers.
    So if all you have is a hammer clearly its the right tool to use.
  10. Ignoring implementation differences such as programmer familiarity and ease of use (which are of course important!), from a protocol perspective IMO the major difference between SNMP and HTTP is that SNMP specifies not only the transport layer but also the structure of the data in the PDUs - when you request an OID the client already knows what the type of the data is from the MIB.

    HTTP does not enforce this coupling of the transport and the message content. Whilst it is possible to request a particular MIME type encoding (i.e. text/html), the format of the response is not defined by the HTTP protocol specification. Of course, the client and server can use a MIME-type which has the ability to use schema to describe the messages (such as XML) - but then you're starting to look more SOAPy and less RESTy.

    I don't know if there are existing standards that provide schemas for defining the messages content - if there aren't any then the future will be painful for anyone who wants to interact with multiple vendors each of whom is using their own schema.

    To avoid this, I would like to propose the intriguing possibility of adding a new MIME type of text/snmp - allowing the HTTP REST'ful approach to be used with the schema and data types of SNMP!

    For example a HTTP GET with the to the URL:
    http://switch1/get_status/oid/1.3.6.1.2.1.1.3

    would return the ASN1.1 representation of the switches system uptime (or perhaps a JSON representation of the same).

    Perhaps this could be made part of the FCoTR alliance?

    PS. After I had this novel idea (to me anyhow), I searched and found this RFC draft: http://tools.ietf.org/id/draft-mellquist-web-sys-00.txt - which is similar in scope, although they just POST the SNMP PDU in the HTTP body don't use the OID in the path of the URL.
    Replies
    1. What on earth would be the point of that? So you can do SNMP (a UDP lightweight procotol) with HTTP, a heavyweight TCP protocol?

      What do you gain?
    2. @Dave: you should do some research on FCoTR ;))
    3. Ah - ok leaving aside that you were joking it raises a serious point. The reason that MiBs work is that there is a large core set that are implemented on almost all systems - which means that the same data gathering mechanism, and same request works everywhere.

      So sure, using REST to gather data would work just fine - relatively inefficient, but fine. But you'd need the entire industry to agree on just how to request the optical signal levels, or mac table from any given device.

      I'm not holding my breath on that one.
  11. Back in my server admin days, I did a lot of work with SNMP, specifically to get server stats and trend them with MRTG and its offspring (like Cactus).

    I can say with all seriousness: SNMP is a terrible protocol, and should die a horrible death.

    SNMP 1 and 2 are very similar, although SNMP 2 has three distinct versions: 2a, 2b, 2c. Only 2c is really used. The biggest difference is that it enabled 64-bit counters. Both 1 and 2x are unencrypted.

    SNMP 3 changed a lot, including adding very crappy encryption (a simple pre-shared key). It was almost a decade before vendors started to implement SNMP v3 in any significant way.

    MIB files are a huge pain in the ass to parse, especially with your eyeball. The tools available are equally as crappy.

    RESTful APIs allow a much easier way to get access to data and do configuration. XML is a shitload easier to parse than a MIB file. And REST allows PKI (SSL/TLS and certificates and such) which offer much better (potential) security.

    Like modem init strings, OIDs haunt my dreams at night. Both are things I hope I never run into again.

    SNMP: Simple Network Management Protocol. The biggest lying acronym in IT :)

  12. How does the REST/HTTP provide similar mechanism to SNMP notifications (traps)?
    Replies
    1. HTTP callbacks. The REST server (in our case the network device) executes a GET or POST request on an URL provided in an API call.
  13. The short answer suggesting that there is no difference between the two because one can do the same thing as the other is a classic logical fallacy. One can, as well, compare FTP file transfer protocol and SNMP, i.e. one can use SNMP to transfer a file, but this would not be the most efficient way to do it, and transferring a files is not what SNMP is good at. This article would serve better purpose if it only explains what SNMP is good for and what it is not good for and what REST is good for and what it is not good for without comparing the two. In a modern equipment, such as a servers, a network controllers, etc., typically you can find both, in ancient equipment SNMP is typically the one and only interface.
  14. All that said, in gNMI (uses gRPC) you can get differential telemetry, that is, get only what has changed. Considering systems with lots of management parameters to take care off (e.g., optical WDM systems), that is very relevant.
  15. You are comparing apples and oranges. Its like saying REST is better than SSH. An argument can be made, but its pointless.

    REST is certainly a hell of a lot more powerful than SNMP. But for the purpose of extracting stats out of boxes, SNMP is amazing because it is cross vendor compatible. I can pull stats from Juniper and Cisco and you name it routers with the same command / set of commands. I don't have to write a separate adapter for every little gizmo in the world.
    Replies
    1. ... and you’re conflating transport protocols (SNMP or REST) with standardized data models (MIBs versus OpenConfig for example). Give OpenConfig/YANG another 20 years...
  16. I know this article is very old by now, but as someone who spent many years working with SNMP, I strongly disagree with the conclusion. SNMP has one (possible) advantage over REST, which is that it is connectionless. After that, it is a horror show.

    The biggest issue is probably the amount of space that OIDs consume in limited-size packets. GetBulk did not end up implementing any OID compression, and so when walking a table of counters, gauges, or states, for example, almost the entire, precious 1500 byte packet capacity is consumed transporting OIDs. In the real world, packet fragmentation and reassembly does not work well, and so you have to carefully craft your request packets to avoid size overruns in responses.

    The only traversal semantic is SNMP is GetNext/GetBulk. There is no filtering, that I am aware of, and scoping is only available by specifying more of the OID prefix, which is limiting.

    Table walk semantics are difficult to implement correctly for sparse tables. Even if you implement that correctly, you have to be careful which other columns your GetBulk runs over into at the end of the table. We had a case where the last row of a table could never be fetched due to running over into an unrelated table with a too-large OCTET STRING. We had to create a custom table-walk algorithm for that specific table.

    Because many agents are single-threaded, you have to implement throttling on the management side to avoid spurious timeouts and retries. (Without throttling, requests queue up in the agent, including retries, and everything times out.)

    CRUD semantics for tables are difficult to implement correctly. At my last job, we almost got sued by a customer because one out of a sequence of SetRequest packets failed, and left a wide-open WLAN on their network. Since SetRequest is not idempotent, you cannot even retry those.

    There are other parts of the protocol I could pick on, such as community strings for authentication, or OID index parsing, or the difficult-to-use InetAddress/InetAddressType data types. There is a lot that did not age well.

    Another commenter stated that SNMP is a terrible protocol that needs to die. I don't know if I'd go that far, but for managing a device of any complexity, it is so much better to implement a REST API. You have reliability and the ability to stream large amounts of data, if needed. Scoping and filtering are available via URLs and query parameters. JSON encoding is not even much more verbose than all of the SNMP OIDs that come back in GetBulk. When you ask for a table of data, you don't get unrelated tables. You don't have to implement "sparse walk".

    REST APIs also allow for various authentication schemes that HTTP supports. TLS is supported simply.

    There are dozens of open source libraries and frameworks for HTTPS, REST, and JSON.

    REST is just superior. Anyone who says "SNMP can do anything that REST does" has, I would wager, never spent much time maintaining a sophisticated SNMP-based management application. I have done that, and I guarantee that the statement that the two protocols are equivalent is naive.

Add comment
Sidebar