“Government information shall be managed as an asset throughout its life cycle to promote interoperability and openness, and, wherever possible and legally permissible, to ensure that data are released to the public in ways that make the data easy to find, accessible, and usable” (emphasis mine).

No Dumping
The usual approach to this type of work is to simply publish raw data in a directory or repository and then create some fencing around the data that helps track usage and distribution. Essentially, making government data “open” becomes a data dumping operation. This practice fails on all of President Obama’s three key points. First, data dumps make finding valuable information not at all easy. Second, even though the content might appear in a standard format like XML, CSV or JSON, it is hardly accessible (except for to geeks, who love this kind of stuff). And finally, raw data is hardly ever usable. Instead, it’s a mind-numbing pile of characters and quote marks that must be massaged and re-interpreted before it comes close to usability.

So, while this new directive offers an opportunity to make available a vast amount of the data the government collects on our behalf, the devil is in the details. And the details are in the interface – the API. As with poorly-designed kitchen appliances and cryptic entertainment center remote controls, when it takes extensive documentation to explain how to use something, the design has failed. There’s a simple principle here. Poor API design results in unusable data.

Affordable Data
It doesn’t have to be this way, of course. Government departments have the opportunity to implement designs that meet the goals set forth in the executive order. They can make it easy for people to find, access and use the data. They can publish not just data but APIs that afford searching, filtering and exploring the data in a meaningful and helpful manner; APIs that empower both users and developers to successfully interact with the data, without resorting to a dashboard featuring dozens of options or mind-numbing explanations.

In the (likely) event that the initial open data release consists of mere data, companies and individuals would be well advised to resist the temptation to build a multitude of “one-off” applications, each of which solves a single problem or answers a narrow set of questions for some subset of the data. Instead, work should be put into converting the raw data into usable API formats such as Atom, OData, HAL, Collection+JSON and HTML (to name just a few). APIs should be designed with the same care that would be given to any interactive experience.  Investment in tools and technologies that can properly represent the data in multiple formats while supporting various use cases and access requirements will yield great results.

Open Data APIs
In the end, organizations that know the importance of a good interface, the power of choice and the freedom of flexible representations will be able to convert raw data into valuable information, which can be consumed by a wide range of users, platforms and devices. These considerations are essential to building and supporting open data APIs.

Because – ultimately – data isn’t open, unless it’s “easy to find, accessible, and usable”.

For instance, most organizations will derive far more immediate payback from an API initiative if it enables internal developers, enterprise mobility initiatives, tighter partner integrations or even IT rationalization through hybrid cloud. Each of these endeavours will pay dividends in terms of productivity, agility, distribution and lowered IT costs. Each deserves its own dedicated discussion. However, underpinning all of these API business drivers  – external developers included – there is one often-overlooked consideration for cost and return in any API program: how do you introduce and innovate new APIs cost effectively?

Obviously, there are many ways to stand up an API. Many packaged software applications have some kind of API already, even if some are XML- or SOAP- centric. But in many instances, nothing exists except the desire to expose a piece of functionality or quantity of data as an API. Programmers can obviously build “programmable  interfaces” onto almost anything. It just takes time and people. However, the results will be brittle and the journey expensive.

A faster, less costly and more flexible route is to use an adaptation layer that can talk to various application or data backends and dynamically render one or more as an API. Using a backend adaptation layer can, with the right product, also solve the related problem of iterating on an API, both in terms of versioning but also composition. Add to that the promise of facilitating new business functionality by orchestrating API interactions with external mobile, social and cloud services and you get a pretty compelling ROI story.

Not surprisingly, Layer 7 provides such an adaptation layer. Our API Gateways provide more than just security and management; they simplify backend connectivity, new API formation (i.e. composition) and novel orchestrations with all kinds of cloud, social and mobile services. Like many of our API compatriots, we provide tools that help enterprises build and foster developer ecosystems. But we also realized early on that much of the cost and potential of an API program will rest on how quickly and cost-effectively new services can be launched and evolved. Something worth considering the next time you evaluate the ROI of an API program.

If They Have to Ask…
Try this:

• Create a new Web API
• Get it up and running on some server or other
• Hand the single URL to a client dev and say: “There ya go!”

Is the API self-descriptive? Does it contain enough information in the responses to allow client devs to know what the API is for, what it is capable of and how they can make valid requests to the server and properly parse the responses?

Here are some questions for you:

• How many assumptions do you have about your API?
• Are these assumptions shared by client devs?
• All clients devs?
• Even ones who have never met you?

If your answer to any of those questions was “No” or “I’m not sure” then it’s likely that devs will need to ask you a thing or two about how to properly use your API. That’s no big deal, right?

…You Didn’t Afford It
In everyday life, if people have to ask how to use a device (television remote, toaster etc.) then you can be sure that device is “poorly afforded” – it’s a case of weak design. We all know devices (especially electronics) that come with huge manuals and complicated explanations – and we all know what a bummer it is when that happens.

In this respect, your API is the same as any other consumer device. It should be “well afforded” – developers shouldn’t have to read the technical equivalent of War & Peace before they are able to successfully use your API.

Yes, you can supply detailed instructions in prose, provide a long list of possible methods, include lots of tables etc. These resources are helpful for devs but they can be daunting to read and cumbersome to maintain.

Another approach is to include this kind of information in a machine-readable format – and one that most devs will also understand quickly. This can be achieved by providing instructions (that get automatically updated whenever your API changes) via hypermedia controls in the response. Why write a Web page of documentation to tell devs to construct a URI and use that URI to execute an HTTP GET when you can just include that (and much more) information in your API responses?

Help your client devs out. Throw ‘em a bone, here. Don’t make them read pages of documentation when you can just include simple run-time instructions as they’re needed.

In conclusion: If they have to ask, you didn’t afford it.

(Originally published on my personal blog.)

Clearly, there is business value to be attained by companies who utilize an API, and an accessible web API is a requirement for modern corporations.  For companies looking to launch an API, there is a temptation to focus on the technological aspects of implementation.  Good API design, architecture, and infrastructure are vital to the success of a company’s API, but there are other areas to address first.  I am currently reading the book “Why Nations Fail”, and recently read “Thinking Fast and Slow” by Daniel Kahneman.  Although the former is a geopolitical study whereas the latter focuses on the human mind, both share an identical observation that is the foundation of their arguments: a great amount of economic study is flawed because it fails to account for human behavior and tendencies.  I feel the same way about technology.

Every paradigm shift in technology has been driven by both innovation—the new technology itself—and application—how that technology can be used.  In other words, there is a machine side and a people side to every technology change.  The technologists responsible for implementing these changes often bias towards their comfort zone—the machine side—and overlook the people side.  This has led to frustration for companies who invest significantly in new technology only to miss the intended benefits of the change.  For APIs, the people side of the change is especially important.  In fact, the social nature of the API world means there are even more groups of people to consider.  Ultimately, the success of a company’s API will depend on the creation of a diverse community for that API—end users, partners, developers, and more—as well as the adoption of a business model that allows the API to contribute to the company’s bottom line.  Taking the community and the economics together, this means you will need to build a nation for your API.

Some of the biggest companies on the web have taken this approach with their APIs, and I recently explored some of their winning tactics in this VentureBeat article.  Please have a read and let me know your thoughts, and perhaps your own API lessons

At the Layer 7 API Academy, we’ve had a few requests from API designers who are seeking strategies for handling large amounts of data in API responses.  Pagination is the most common method for addressing this scenario. Pagination, which is very common on the Web, allows API architects to conserve resources, improve response times and optimize the user experience. It’s a way of splitting up data into “pages” and is used in just about any API that returns collections of data.

I’ve released a short video tutorial titled Use Pagination in Web API Design to introduce the ins and outs of the interface. This video provides a crash course explaining pagination and outlining how to use it effectively in the design of Web APIs. I couldn’t fit all the implementation considerations I wanted in this six-minute tutorial, so watch out for a follow-up video on the subject.

Each technology, service or coding framework exhibits its own “style” for solving a problem. Sometimes we select a system component because it’s familiar (“We use SQL databases because that’s what we’ve always used”). Sometimes we include one because it’s unfamiliar (“We’ve never used Node.js before, let’s try it on this project”). And sometimes we select components based on skill set (“Our team doesn’t have any experience with WebSockets, so let’s just use HTTP instead”). It’s important to step back and get a big picture view when selecting components for a production system that will (hopefully) serve your needs for an extended period of time. And that’s where architectural styles come into play.

Architectural styles set the tone for how components in a system interact, govern the implementation details and establish lines of responsibility and maintenance over time. Setting the style early on and communicating it to the team ahead of time goes a long way toward creating a stable and successful implementation. To help clients get a handle on this topic, I commonly identify three widely varying-styles for Web solutions that people can easily recognize: Tunneling, Objects and Hypermedia.

The Tunneling style is best illustrated by SOAP-based implementations where all requests are “tunneled” through a limited set of components (user management, product services etc.) exposed on the Web. The Object style is one that uses the HTTP CRUD pattern (create-read-update-delete) where domain objects (users, products etc.) are exposed and basic read/write operations are supported for those objects. The Hypermedia style relies on a shared understanding through a message format (media type) that defines both the data elements (users, products etc.) and the possible actions (read, write, filter, report etc.) on those data elements. Each of these styles can be used to implement a solution and each of them has associated benefits and challenges.

This comes up so often that we’ve created a short API Academy video introducing the subject of architectural styles for the Web. Take a look and see if it gives you some ideas for how you can answer this question the next time you are about to embark on a major system implementation: “What architectural style is appropriate for this Web solution?”

That’s why we at the API Academy stress the importance of making rational decisions rather than irrationally selecting design patterns based on emotion or trends.  We want you to choose your design elements rather than picking them from the latest set of formats or technologies that you’ve heard about.

And that’s why we’re working on a series of tutorial videos, as my colleague Mike Amundsen recently announced. The first of these videos, titled The API Interaction Model – An Introduction, provides an overview of  a design process that will help you consider your user’s perspective in order to make effective design choices later. The ideas I discuss in this video are rooted in user-centered design processes that have been very effective in the software and product design worlds.

If you’re currently designing an API, invest five minutes and watch the video.  It should be time well spent.

This time around, Mike will be talking about Applying the USE Paradigm When Designing Your APIs.  The USE (Usable, Scalable, Evolvable) paradigm provides an extremely useful guide when creating high-level interfaces for your existing business objects and data storage.

Mike will be taking live questions on the USE paradigm and API design generally, so get ready to get involved. Tweet questions with the hashtag #layer7live or email techtalk@layer7.com. On the day, you can watch and chat live on the Tech Talk page.

This looks set to be a very special Tech Talk for anyone interested in API architecture and design – so don’t miss out; join us on Tuesday!

I also point out that using a hypermedia-based approach to building client and server applications takes a different kind of effort than using RPC-style approaches. And I explain that, currently, there is a limited amount of tooling available to support the process of designing, implementing and maintaining hypermedia-style systems.

If your work involves designing, building, testing and deploying a mobile client application, it is likely you need to deal with an “application store” or some other process where your packaged application must be submitted for review and approval before it is available to users for download. This can happen not only with well-known public offerings such as the Apple Store but also within any organization that provides its own application repository aimed at ensuring the safety and consistency of user-available mobile apps.

In an environment of quick-turnaround, agile-style implementations this “app store” approval can be a real bottleneck. It may be not just days but weeks before your app is tested, approved and posted. This can be especially frustrating when you want to deploy a rapid-fire series of enhancements in response to an engaged user community.

A hypermedia-based client design can often support UI, data transfer and workflow modifications by altering the server messages rather than altering the client source code. By doing this, it is possible to improve both the user experience and the system functionality without the need for re-submitting the client code for “app store” review and re-deployment. This also has the potential to reduce the need for interrupting a user’s day with download and reinstall events and can, in the process, cut down on the bandwidth costs incurred during the repeated roll outs of code modifications to a potentially large user base.

Improved agility, a better user experience and reduced bandwidth costs are all tangible benefits that are possible when investing in a hypermedia-based implementation for your mobile client application.

I’m happy to announce the release of the first API Academy video shorts. I’ve been working with my colleague Ronnie Mitra to create a series of short (five-minute), informative videos on topics related to the Web, APIs and solution design/implementation.

These first few videos are just the start. We plan on doing more of these shorts on a wide range of topics, over the coming weeks and months. And we need your help. Please take a look at these first vids and send us your feedback.

You can comment here, on YouTube or by emailing me directly. We’re looking for feedback on the format, suggested topics and even how we could improve upon this model (hosting a separate site, adding interaction, badges etc.)

Any time you can spend on watching these and sending comments will be most appreciated. Our aim is to do something helpful, engaging and – above all – enjoyable. Thanks for your help and let’s see what this can become!

The API Interaction Model – An Introduction

—

Three Common Web Architecture Styles

—

Handle Errors on the Web