November 29th, 2013

Ending the IoT Protocol Wars

Ending the IoT Protocol WarsIt’s been a while since my last blog post – not least because I have been traveling quite a bit to run Layer 7’s European API workshops together with my colleague Ronnie Mitra. The workshops (part of Layer7′s outreach program via the apiacademy.co) are vendor-neutral and focused on sharing API design and management best practices.

To be honest, I probably learn as much during these workshops as the participants do. It has certainly been striking to watch how our material evolves throughout the workshops. We constantly keep adding and tweaking material, based on what we learn. In particular, I’m struck by the amount of changes my IoT section has been going through.

Here is what I have learned regarding IoT protocols: It’s a zoo out there, with lots of protocols trying to become the next HTTP. And some candidates deploy a formidable array of marketing, making it exceedingly hard to cut through the fog.

My current shortlist of main contenders is (in alphabetical order):

I might add STOMP to that list, just for its simple brilliance. STOMP is a text-based messaging protocol that has recently been extended to allow for binary content. Additionally, I’ve recently started talking with some transportation companies and learning about their use of DDS, which might be another candidate for the shortlist.

In the corner of residing champion, we have JSON/HTTP. Not content to see this protocol pushed into early retirement, advocates have been developing some very interesting approaches that attempt to ensure the continuing relevance of HTTP for asynchronous small messages – WebSocket being the most well-known. Hypercat, Simple Thing Protocol and EventedAPI represent just a small sample of the interesting approaches emerging to support async eventing and messaging with HTTP.

Where does this leave a developer trying to choose the right protocol for that awesome winged steam punk toaster? I don’t really have the answer but there are some documents trying to tease out the differences. Take a look at the MQTT vs. CoAP comparison from 1248.io or the DDS/AMQP/MQTT/JMS/REST comparison from DDS champion PrismTech.

Based on what I’ve learned so far, only XMPP and DDS have significant commercial deployments while MQTT is being evaluated by almost every major vendor I have talked to. While MQTT’s use as the protocol powering Facebook’s messenger is a good demonstration of its scalability, I don’t think this constitutes a proof point for mission-critical commercial deployments. If you know of commercial deployments of MQTT, I’d love to hear about them.

Each protocol has weaknesses: MQTT appears to be weak in security; DDS seems to be complex to scale and has version dependencies; XMPP is considered heavy-weight. But they all have strengths too, of course: DDS has the deployments in the field to prove its relevance; XMPP supports EXI and WebSocket for efficiency and a proven track record; both DDS and XMPP are extremely mature and have built-in security. Given the industry interest in MQTT, I am sure that whatever security problems exist will be fixed in one of the next versions. The one puzzling piece is the absence of CoAP in a commercial deployment. Again, if you know of one, please let me know.

Where do I stand on all of this? Having watched technologies rise and fall, I think it’s very normal at this stage to have multiple contenders trying to improve on HTTP. What I try to keep in mind though is that both bandwidth and computing power seem to be on an ever-increasing trajectory, while at the same time becoming cheaper and cheaper. Reduction in power consumption and increase in battery capacity, mostly driven through mobile, further lowers the bar for mainstream technology to power even small devices. I would not be surprised if, after the initial phase, we continue to see HTTP and JSON being dominant. As geeks, we sometimes get too excited about efficiency gains while losing sight of the fact that, for most products, technology simply needs to be good enough. But I won’t complain if I am proven wrong this time.

And don’t just take my word for any of this. To help you learn more, here are a couple of other articles reviewing IoT protocols:

August 3rd, 2012

Standards, APIs & WAC

Wholesale Applications Community LogoGigaOM recently ran a piece opining the demise of the Wholesale Applications Community (WAC) after only a couple of years on the scene. The article complained that something like the WAC effort is needed and suggested that, given the nature of the industry and the players involved, it’s not likely to happen. However, what the author failed to notice was that the WAC’s attempted solution was way off the mark.

The WAC’s key failure was that it attempted to standardize the wrong thing: the API. This is a common problem that occurs repeatedly. GigaOm readers may recall another example of industry-level standards going astray, summarized in the “Cloudstack-Openstack Dustup” piece from April. I suspect several readers can call to mind similar cases in the not-too-distant past. Such cases usually share a common theme: disagreement on the details of the API.

The solution is right at hand but few see it. The right way to go is to standardize the way messages are designed and shared, not the data points and actions themselves. In other words, the key to successful shared standardization is through media-types and protocols. This is especially true for any communication over HTTP but it holds true for standards operating over any application-level protocol.

We don’t need to look too far to see an example of an industry-led standardization success. VoiceXML was started by AT&T, IBM, Lucent and Motorola as a way to standardize interactive voice system communications. Not long after the first markup was defined in 1999 (a process which took a matter of a few months), the standard was turned over to the W3C for continued growth and refinement.

The goals of VoiceXML were strikingly similar to those of the WAC and Cloudstack/Openstack efforts: defining an interoperable standard that could be used across an industry group. The difference in the case of VoiceXML was that the committee focused on message design and domain-specific details shared by all players. It did not attempt to document all the data elements, function calls and workflows to be used in lockstep by all.

Most likely, the WAC meltdown won’t be the last one we’ll see. But this is not the inevitable result of competing interests in the global marketplace. This is a result of well-meaning people aiming at the wrong target. We can do better. We can learn from successful interface designs and focus on making it possible to consistently communicate a wide range of information freely instead of attempting to constrain systems to a single set of possible interactions.

The future of an effective Web, a growing and vibrant distributed network, rests in the hands of those who would take on the task of writing the vital standards that will make it work. I look forward to seeing more efforts where the focus is on improving communication between parties through well-designed message formats instead of on limiting communication though constrained APIs.