December 23rd, 2013

Thanks to All Who’ve Been Good This Year

Layer 7 Holiday Promo 2013The year 2013 has been one heck of an adventure for me. My work with Layer 7, CA Technologies and the API Academy team (yes, we have many names!) has taken me around the world, allowed me to speak at several amazing conferences and provided the chance to interact with some remarkable organizations working on APIs for the Web and enterprise. Along the way, I’ve met many incredibly smart and generous people.

In the last year, I’ve worked with organizations striving to reinvent the role of the enterprise architect from a controlling force to an enabler – a person who ensures the development environment is a safe place to be creative; a person who provides help to product groups and development teams via research and guidance taken from a wide range of sources; someone who works to empower teams and cut down on unneeded ceremony and red tape. These are good people and they’ve been a pleasure to work with and learn from along the way.

I’ve also met many conference organizers and community leaders doing essentially the same thing from a different angle. Along the way, I’ve met people who are devoting huge chunks of time, effort and resources to creating events that improve communication, facilitate collaboration and foster success across a range of communities. It’s been really amazing to be a part of these events and to meet so many giving and open people working toward a common goal.

My experience online has been equally enlightening. In the last year, I’ve “met” many new and interesting people, discovered several helpful efforts and organizations. I am lucky that I can learn something new every day online from those I’d likely never meet in person, simply because we are physically far apart.

One experience in particular has marked 2013 for me. I had the honor to work closely with Leonard Richardson on a book project – RESTful Web APIs. It was Leonard’s idea to create the book and I was happy he invited me to help shape the message and content. I’ve learned a great deal from him and I can see the results of that work in online comments and reviews. I am pleased to be associated with Leonard’s talent and vision.

There’s a common thread through all these experiences: I’ve had the luck and privilege of meeting many “good” people this year. This blog post is my way to give a blanket shout out to everyone who challenged me, taught me, invited me, supported me and hosted me in so many ways in the last year. Thanks!

As another small way of saying thanks, we’re offering several free copies of the RESTful Web APIs book to some of those who’ve been “good” this year. All you need to do is add yourself to our “nice list” (go ahead, you know you deserve it). We’ll be giving away a couple dozen copies of the book soon after the holidays.

So, again, thanks to all for your help and support in 2013. And look out for us in 2014 – things are just getting started!

December 2nd, 2013

How I Lost Weight & Learned About APIs

How I Lost Weight with APIsTrying to stay in shape is one of those never-ending life battles that I’ve come to expect as I get older. I’ve bounced between being a healthy shape and a not-so-healthy one for years and I’ve managed to live life just outside the edge of ideal fitness. A few months back, I reached an apex point and dedicated myself to losing a few pounds (again) and set off on a journey to change my life (yet again). Little did I know I’d learn something about APIs along the way.

Everyone has their own way of losing weight but I’ve always preferred a measurable, rationalist approach: I count the calories I consume, I subtract the calories I burn and I budget accordingly. The nice thing is that this method forces me to think about what I’m consuming and what I’m doing. The massive downside is that keeping track of all of the data is a monotonous and soul-destroying effort that often leads to me giving up.

Of course, there is an app for everything now and I started using  a tool to keep a log of foods that I ate along with their associated caloric burdens. One problem with this type of tool is that, while it’s easy to log consumption of food using features like bar code scanners and crowd-based data, the process of logging exercise and calorie expenditure is entirely manual. This can make fitness goals harder to achieve as users like me end up either under or over estimating their daily calorie burn.

Thankfully, devices to monitor your physical exertion do exist and they are reasonably affordable. These are wearable devices that provide a tally of steps taken, stairs climbed and physical exertion throughout the day, providing a wealth of personal data to mine. To be honest, I’d always viewed these devices in the same category as things like Google Glass – really cool pieces of technology that bleeding-edge enthusiasts wear publicly at the cost of their own dignity. But something changed for me when I realized that I’d be able to connect the calorie-counting app I was using with the wearable fitness device. So, I made a purchase.

By connecting the food-tracking application with the activity-tracking device, I was able to get a much more accurate picture of my caloric budget for the day. The systems integrated remarkably well and the quantification of remaining calories along with a few gamification features provided extra incentive for me to keep moving and eat less.

In the end, this behavioral conditioning of triggers, alerts and feedback loops worked well for me and I was able to drop a few pounds. Of course, I lost the tracker on an airplane about a month in and I’m currently racing back towards a pear shape but that isn’t the point. What is more interesting is what we can learn about integration from my journey:

1.  An API is a Great Way to Extend Customer Reach to Platforms
When we think about building APIs, we usually think about extending out to mobile devices or social platforms. But organizations should consider how their products can be extended to niche and non-traditional platforms that their target user base actively uses. If the wearable tracker I purchased didn’t work with the calorie-counting application I was already using, I never would have considered buying the tracker in the first place. But thanks to the API-based integration, I could visualize myself using it and this was the trigger that resulted in a purchase decision.

2.  Integration is Becoming a Core Requirement Instead of a Feature
Something I noticed when scanning the forums on the tracker device’s Web site was the number of posts related to integration with other exercise platforms. For this user base, integration with their favourite run-tracking, calorie-counting or fitness-gamification tools isn’t just a nice-to-have – it is the minimum expectation. It seems that end users are increasingly expecting product vendors to support their platforms of choice and want the freedom to make their own decisions. In other words,  users don’t want to be punished for choosing a less popular tracking tool or a mobile phone operating system that has less market share.

3.  Integration with Potential Competitors can Pay Off
What I didn’t mention in my story was that the fitness tracker I purchased did come with a calorie-consumption-tracking feature. In fact, part of the revenue stream for this product is the sale of subscriptions to the manufacturer’s fitness portal, as part of an end-to-end fitness management program. This means that supporting out-of-the box integration with other fitness trackers actually comes at a potential revenue cost for the tracker vendor. But I would imagine that the overall revenue benefit from attracting customers like myself outweighs the revenue lost from users who choose not to subscribe to the portal. Integrating with competitive products can be a risky proposition but a smart gamble can really pay off.

As interest in the Internet of Things (IoT) continues to increase, I expect to see an increasing variety of interesting device-to-platform integration stories. Businesses will need to have coherent business strategies for extending to this new world, with APIs as an important supporting action.

Also, if you happen to see me in person, don’t forget to tell me how great I’m looking nowadays.

November 29th, 2013

Ending the IoT Protocol Wars

Ending the IoT Protocol WarsIt’s been a while since my last blog post – not least because I have been traveling quite a bit to run Layer 7’s European API workshops together with my colleague Ronnie Mitra. The workshops (part of Layer7′s outreach program via the apiacademy.co) are vendor-neutral and focused on sharing API design and management best practices.

To be honest, I probably learn as much during these workshops as the participants do. It has certainly been striking to watch how our material evolves throughout the workshops. We constantly keep adding and tweaking material, based on what we learn. In particular, I’m struck by the amount of changes my IoT section has been going through.

Here is what I have learned regarding IoT protocols: It’s a zoo out there, with lots of protocols trying to become the next HTTP. And some candidates deploy a formidable array of marketing, making it exceedingly hard to cut through the fog.

My current shortlist of main contenders is (in alphabetical order):

I might add STOMP to that list, just for its simple brilliance. STOMP is a text-based messaging protocol that has recently been extended to allow for binary content. Additionally, I’ve recently started talking with some transportation companies and learning about their use of DDS, which might be another candidate for the shortlist.

In the corner of residing champion, we have JSON/HTTP. Not content to see this protocol pushed into early retirement, advocates have been developing some very interesting approaches that attempt to ensure the continuing relevance of HTTP for asynchronous small messages – WebSocket being the most well-known. Hypercat, Simple Thing Protocol and EventedAPI represent just a small sample of the interesting approaches emerging to support async eventing and messaging with HTTP.

Where does this leave a developer trying to choose the right protocol for that awesome winged steam punk toaster? I don’t really have the answer but there are some documents trying to tease out the differences. Take a look at the MQTT vs. CoAP comparison from 1248.io or the DDS/AMQP/MQTT/JMS/REST comparison from DDS champion PrismTech.

Based on what I’ve learned so far, only XMPP and DDS have significant commercial deployments while MQTT is being evaluated by almost every major vendor I have talked to. While MQTT’s use as the protocol powering Facebook’s messenger is a good demonstration of its scalability, I don’t think this constitutes a proof point for mission-critical commercial deployments. If you know of commercial deployments of MQTT, I’d love to hear about them.

Each protocol has weaknesses: MQTT appears to be weak in security; DDS seems to be complex to scale and has version dependencies; XMPP is considered heavy-weight. But they all have strengths too, of course: DDS has the deployments in the field to prove its relevance; XMPP supports EXI and WebSocket for efficiency and a proven track record; both DDS and XMPP are extremely mature and have built-in security. Given the industry interest in MQTT, I am sure that whatever security problems exist will be fixed in one of the next versions. The one puzzling piece is the absence of CoAP in a commercial deployment. Again, if you know of one, please let me know.

Where do I stand on all of this? Having watched technologies rise and fall, I think it’s very normal at this stage to have multiple contenders trying to improve on HTTP. What I try to keep in mind though is that both bandwidth and computing power seem to be on an ever-increasing trajectory, while at the same time becoming cheaper and cheaper. Reduction in power consumption and increase in battery capacity, mostly driven through mobile, further lowers the bar for mainstream technology to power even small devices. I would not be surprised if, after the initial phase, we continue to see HTTP and JSON being dominant. As geeks, we sometimes get too excited about efficiency gains while losing sight of the fact that, for most products, technology simply needs to be good enough. But I won’t complain if I am proven wrong this time.

And don’t just take my word for any of this. To help you learn more, here are a couple of other articles reviewing IoT protocols:

October 16th, 2013

Intelligent APIs for Big Data & IoT

Written by
 

Big Data Webinar“Data is the new oil” is an oft-repeated phrase. But when was the last time you went out and bought a barrel of crude oil?  The value to consumers is in the refined product: gasoline. With data, the refined product is information – the distilled and actionable essence of multiple sources of raw data.  So, if “data is the new oil” then “information is the new gasoline”.

There’s a lot of data out there and IoT is going to increase it greatly. For large organizations, refining Big Data stores is a significant challenge. This is partly because data doesn’t start out big but gets collected from lots of relatively small sources. Also, data seldom arrives in the right format for sharing and monetization. Furthermore, responsibility for securing and managing data is not always in the same hands as responsibility for sharing data.

We have explored some of these issues in recent blog posts like Was is DaaS? and How APIs Grease the Data Wheels. In tomorrow’s webinar, Intelligent APIs for Big Data & IoT, Matt McLarty and I will try to bring it all together and talk about how APIs are becoming the pipelines and tankers that move the gasoline from its source to the user.

October 7th, 2013

SDKs or APIs: What’s the Right Choice for Your Developer Community?

SDK or APIWhen creating an application programming interface (API) for a service, one of the key decisions any program or product manager will face is how best to meet the needs of their prime target audience: developers. Faced with this decision, you want to make sure your API is easy to use and doesn’t represent a high barrier to entry for your specific developer audience.

Currently, the typical approach is to design an interface that leverages the most common protocol on the Web today: HTTP. This is often labeled a “RESTful” API (referring to Roy Fielding’s architectural model for the World Wide Web) and offered as a one-size-fits-all (OSFA) model for developers to use when building client applications for a service. But this is not always the best approach.

Properly understanding and implementing a raw HTTP interface may be too complex for some segments of your developer community – some of whom are really only interested in your service and not in spending time to build a killer HTTP application for that service. Additionally, a generic HTTP API might end up compromising key performance aspects of your service in order to work best for a wide range of developer communities (the OSFA problem). Even worse, an HTTP-based API may – in the end – result in unfocused client applications built by developers who know more about HTTP than your service.

What We Learned from SOAP
One of the powerful lessons learned from the SOAP community relates to the value of developer tooling. The SOAP Web Service Definition Language (WSDL) is a complex, difficult-to-read document that contains all the important details on building a compliant client application for a service. And developers have a hard time making sense of the document. To solve this problem, the SOAP community helped promote an “accommodation” for developers –  the “WSDL” button that is available on many code editors. By simply pressing this button and following a few prompts, developers can easily create API facades for servers or consume WSDL documents to build client applications. The WSDL accommodation makes SOAP programming not only easy, it adds to the usability of SOAP interfaces and lowers the bar for developers wanting to use services on the Web.

What We Lost with HTTP CRUD
The rise of JSON-style HTTP CRUD (Create-Read-Update-Delete) programming interfaces meant the loss of the WSDL accommodation. Developers were expected to hand-craft both the server and client interfaces without the aid of a unifying definition document like SOAP’s WSDL. To make up for this loss, server coding environments like Ruby on Rails introduced helper functions (e.g. “rails new”) designed to generate a great deal of the API facade required in order to publish a service on the Web. Developers could use this accommodation to simplify the interface details and allow them to focus on crafting the internal object and business modeling needed to make the service operational.

Client devs needed to create their own accommodations, too. That’s why we have client-side libraries like ember.js, backbone.js, angular.js and others. Like the developers building servers, client developers needed help handling the basic plumbing for HTTP-based applications, so they could focus on their own object models and business logic.

The bad news is, in this HTTP CRUD world, each and every service has its own unique set of URLs, objects (users, products etc.) and actions (approve, remove, edit, apply etc.). And each API looks like a snowflake among hundreds of other unique interfaces. For example, there are more than 500 unique APIs for supporting shopping services. This can raise the bar for developers and lower the usability of HTTP CRUD APIs.

Adapter APIs
Netflix set out to solve the OSFA problem in 2012 by embracing differences in its developer community and creating a set of targeted “adapter APIs”. These are custom interfaces optimized for selected developer communities. For example, Netflix offers one API for its XBox community and a slightly different API for its PS2 community. In fact, each major device has its own custom API.

What’s interesting about the Netflix approach is that the customized interface accommodations live on the server, not the client. In other words, Netflix has taken on the task of optimizing its interfaces for each community and hosting that optimization on its own servers. A client developer will still be using an HTTP API but it will be one tailored to a specific device – a server-side custom library.

Do SDKs Provide the Answer?
Another way to provide solid accommodations to your HTTP developers is to create software developer kits (SDKs) for your service. Essentially, SDKs provide the same accommodation that WSDL-generated client code does for SOAP interfaces. The good news is that well-designed and executed SDKs can lower the bar for developers and increase service usability.

Recently, Evernote announced that it was taking the SDK approach. One reason for this was Evernote’s decision to use the Apache Thrift message model. The Thrift approach serializes messages using a custom binary format and then ships them across the network using one of a handful of transport protocols (including direct TCP/IP). This is pretty low-level stuff and can easily intimidate some client developers – raising the barrier to entry for Evernote’s API – and this is where creating an SDK is a handy approach. Evernote has committed to building a wide range of language-specific SDKs that clients can download, install and use in their own code. This accommodation lowers the bar when using the Thrift model.

The bad news is that creating SDKs is often a recurring additional expense for your services team. Unlike the WSDL standard, which makes it possible to generate code for a wide range of programming languages from a single published definition file, SDKs usually need to be hand-built for each target programming environment. And selecting target programming languages can turn into a slippery slope. Which languages are most used by your current API consumers? What if the most-used language represents only 30% of your target audience? How many SDKs do you need to build and maintain before you reach a significant portion of your developer community?

The maintenance of SDKs can be substantial, too. How often do you release updates? Fix bugs? Provide new features? Each release can mean added cost to your developer community as they need to open up their own code, integrate the new SDK, run tests and finally re-release their apps to their own target communities. Unless carefully done, this can result in added churn and expense all around – including deployment and download costs for end users. And that can raise the barrier to entry and lower usability.

So What’s a PM to do?
When you start to think about the notion of creating accommodations for target audiences, you have a new metric for assessing the usability and value of your service interface. If you can design an API that is easy for your target audience to use, then you (and your developers) win. If your API is too complex for your audience or relies on a less-used technology, you likely need to include more direct accommodations for your developers.

In some cases, the audience will know how to use SOAP interfaces and will benefit from you offering a WSDL as their accommodation. In other cases, the target audience won’t want/need a SOAP interface and a well-crafted HTTP API (or possibly a set of targeted adapter APIs) will be the right choice. Finally, some target developers will want/need an SDK to handle the protocol details and allow them to focus on their own business logic.

In fact, sometimes the best bet is to offer more than just one of these options in order to reach all your target audiences. Your enterprise partners may prefer a SOAP interface, your mobile devs may prefer a well-design HTTP CRUD API and your business and marketing team may prefer an SDK that exposes only the parts of your service they need to use.

You Own the Interface
The key point in all this is that you own the interface. You can cast your service in many different ways, aimed at several different audiences. You don’t need to stick with a OSFA approach and you don’t have to rule out major technology sectors like SOAP just because a portion of your audience prefers one interface style over another.

By focusing on your target audience and learning their skills and preferences, you can identify key metrics that can drive the selection of the right mix of API styles and accommodations that will help you meet your goals for API reach and usability.