The challenge is that it’s really difficult to find people who are as great at documenting systems as they are at designing them. As a convenient shortcut, many API designers use tooling to auto-generate documentation. This often means exporting machine-readable interface description files like WSDLs and WADLs based on some type of configuration data entered into a development or testing tool. Assets like these are great for driving programmatic components on both the client and server side but they have limited value otherwise.

WSDL files, in particular, are popular in the SOA space because they allow client developers to auto-build proxy classes that can be invoked in the RPC style that is prevalent for SOAP-based integration. This advantage is diminished in the HTTP API world as we have moved away from this document binding style of interface to less structured forms of integration. But even putting this distinction aside, WSDLs have never provided an effective means of documenting SOAP systems and WADLs are no better.

Effective documentation implies effective communication. In the vast majority of situations, a standards-based XML description of your interface is not going to cut it. Application developers need to understand much more than the names and parameters of your API if they hope to build real applications in reasonable time-frames. This means you need to create documentation fit for developers – in other words, documentation built for humans.

Your documentation will act as a navigation system through the complexities of your API. Simply providing a WADL is the equivalent of providing a set of GPS coordinates to a tourist in your city. With the right tools, they may get there eventually but a human-readable map would provide a much richer and simpler experience. If you care about the developer experience for your API, you’ll spend some time and effort writing documentation that works.

“Say you have an end point for validating address; call it /validateAddress. Now this endpoint is called from two work flows.

1. When a user updates his account settings (changes a new address)
2. When a user tries to buy a product and enters the shipment address

In both cases the /validateAddress should give different set of links and forms as part of the response of validation (next step affordances) because the flow is different. In this case what is the set of the next links and forms returned from the endpoint? Is it the union of the two workflows and the client knows how to get what it needs? Or does the client send information of which flow it is in and the server uses the information to figure out what response to give it?”

Decoupling Backend Processes from Public URIs
This kind of question comes up frequently. Essentially, there are a couple assumptions here that are worth exploring. The first is the idea that a backend operation (e.g. “validateAddress()”) is exposed over HTTP as a single endpoint, no matter the calling context. This is not a requirement. In fact, it is advantageous to decouple public addresses (URI) from private operations on the server. HTTP (whether using HTTP-CRUD, Hypermedia-REST or some other model) offers the advantage of using multiple public URIs to point to the same backend operation. For example, it is perfectly correct to publish both /validateExistingAddress and /validateNewAddress URIs each of which points to the same  “validateAddress()” operation on the server.

Not Everything Needs a URI
Just because the backend server has an operation such as “validateAddress()” does not mean there has to be a URI associated with that operation. For example, the “user updates his account settings” workflow need not have a direct URI call to “validateAddress()”. Instead, there could be an account settings resource (/account-settings/) that supports the HTTP.PUT method and accepts a body containing (among other things) a modified address. Executing this client-side operation (PUT /account-settings/) passes data to the server and – along with other operations – the server calls the “validateAddress()” operation itself and reports the results to the client.

The same can be done in the case of “user tries to buy a product and enters the shipment address”. This address validation could be a small part of the server-side operation and processing of an HTTP.POST to a /check-out/ resource.

Mapping Actions to URI & Method
In the HTTP-CRUD model, the focus is on using URIs to identify entities and/or operations and using the protocol methods to perform actions. For example, an /addresses/ resource that supports adding (POST), modifying (PUT), removing (DELETE) and retrieving (GET) addresses associated with a context (logged in user, check-out processing etc.) In this case, POSTing or PUTing a resource body to the server allows the server to call the “validateAddress()” operation (among other things) and report results to the client.

Mapping Actions to Hypermedia Controls
In the hypermedia model, actions are described using a hypermedia control such as a link or form. The URI is not important in this model. Instead the control has an identifier (e.g. “validate“), indicates a protocol action (“POST“) and lists state data to include in the payload.

In Siren it might look like this:

"actions": [
 {
     "name": "validate",
     "title": "Validate an Address",
     "method": "POST",
     "href": "...",
     "type": "application/x-www-form-urlencoded",
     "fields": [
           { "name" : "Street", "type" : "text", "value" : "123 Main Street" },
           { "name" : "City",   "type" : "text", "value" : "Byteville"},
           { "name" : "State",  "type" : "text", "value" : "MD" },
           { "name" : "ZIP",    "type" : "text", "value" : "12345"}
     ]
     }
 ]

Note that I didn’t bother to enter a value for the href in this example. It could be any valid URL; I just left it out.

Tracking Workflow Progress Within Messages
Here’s another question from Abiel Woldu’s email:

“The concept of which work flow the client is going through – is it code that should reside in the API code itself or it’s something that sits outside in some other gateway or something?”

When implementing processes over HTTP, it’s wise not to rely on stateful multi-request chains. In other words, don’t expect either the client or server to keep track of where some request belongs in a workflow. Instead, include that information in the request and response bodies themselves. This pattern of including all the important context information with each request and response not only assures that the request can be handled independently (e.g. in a load-balanced cluster), it also helps clients and servers to do work within varying time-spans (e.g. a clients can cache the last request to disk and pick things up a day later). In the REST model, Fielding described this as making messages “self-descriptive”.

For example, there might be a use case that prompts human users to provide quite a lot of information (across various UI tabs) before finally submitting this completed set of work to the server for final validation and processing. One way to support this over HTTP is to allow clients to store “work-in-progress” (WIP) records on the server. As each “tab” (or other UI affordance) is completed, the client app is free to execute a POST or PUT operation with the payload to a URI supplied by the server. The stored data would include a value that indicates how far along in the workflow the user has progressed. This same client app could also recall stored WIP records, inspect the workflow indicator and prompt the user to pick up where she left off. Once all the required elements were supplied, the work could be forwarded for final validation and processing.

Dynamic Workflow via Hypermedia
Finally, in some cases, the series of steps in a workflow might vary greatly at runtime. For example, a service might support a multi-tenant model where each instance of “supply all the details for this work” has different steps or the same steps appear in differing order. The “next step” need not be memorized by the client code. Instead, hypermedia servers can inspect the current server-side configuration, check the current progress by the user and then supply the correct “next step” for this particular instance.

In this way, the client app can support a wide range of workflow details without needing custom code ahead of time (or even downloaded code-on-demand). Instead, the client app only needs to be able to recognize the “next step” link and navigate to that resource.

In Summary
In general, when using HTTP:

1. There is no rule that you must expose internal methods as public URIs
2. You may use more than one URI for the same backend operation
3. In the HTTP-CRUD model, you usually map operations by linking URIs and methods
4. In the hypermedia model, you usually map operations by linking controls and state variables
5. It is best to use “self-descriptive” messages to track workflow progress statelessly
6. The hypermedia model supports dynamic workflow progress using the “next step” link pattern

Thanks to Abiel for his questions and his generous permission for me to use his email and name in this blog post. If you’ve got a question that I haven’t answered online before, feel free to ping me via twitter (@mamund) and fire away.

We know that connectivity and message-based integration are not new concepts for the financial industry. Enterprise architects in the banking world are masters in the art of connecting old systems with new facades and exposing backend resources through multiple channels. But up until now, a conversation with these professionals would be dominated by concerns rooted in the Services Oriented Architecture world: “How do we prevent service proliferation?”, “How do we secure SOAP conversations?”  In fact, it wasn’t too long ago that the mere mention of an API as a strategic initiative would leave these enterprise architects scratching their heads.

Fast forward to today and many banking technologists are explicitly asking for API Management solutions.  They know the terminology of the space, they know what they want to achieve and they know the pitfalls they wish to avoid. In addition, I’ve been amazed at the depth of knowledge that has emerged among these teams, as enterprise developers have invested their own time to learn from relevant tutorials , articles and – of course – blogs posts. The caricature of the enterprise developer as a SOA dinosaur struggling to understand the “new stuff” is fast becoming a myth.

To be fair, there is still great hesitation within the industry when it comes to opening up data and losing control of the user experience but that isn’t stopping banks from applying good API design practice internally. As we’ve said before, APIs are not simply about reaching  anonymous third-party developers. Indeed, organizations can gain great benefits by applying API Management to the interactions that take place in private for their own apps and partners.  But I’ve been astounded by the handful of architects in the banking world I’ve met who are actively experimenting with public API releases.  It seems that the fear of losing control over data, services and products is beginning to lose out to the perceived value of growing the business through a new channel.

Banks provide a great indicator for the direction of enterprise technology and it certainly seems that the “API thing” has legs within this space. If you are in the enterprise world, make sure you have considered how launching an API program might help your business – because it’s increasingly likely your competitors are already doing so.

“Government information shall be managed as an asset throughout its life cycle to promote interoperability and openness, and, wherever possible and legally permissible, to ensure that data are released to the public in ways that make the data easy to find, accessible, and usable” (emphasis mine).

No Dumping
The usual approach to this type of work is to simply publish raw data in a directory or repository and then create some fencing around the data that helps track usage and distribution. Essentially, making government data “open” becomes a data dumping operation. This practice fails on all of President Obama’s three key points. First, data dumps make finding valuable information not at all easy. Second, even though the content might appear in a standard format like XML, CSV or JSON, it is hardly accessible (except for to geeks, who love this kind of stuff). And finally, raw data is hardly ever usable. Instead, it’s a mind-numbing pile of characters and quote marks that must be massaged and re-interpreted before it comes close to usability.

So, while this new directive offers an opportunity to make available a vast amount of the data the government collects on our behalf, the devil is in the details. And the details are in the interface – the API. As with poorly-designed kitchen appliances and cryptic entertainment center remote controls, when it takes extensive documentation to explain how to use something, the design has failed. There’s a simple principle here. Poor API design results in unusable data.

Affordable Data
It doesn’t have to be this way, of course. Government departments have the opportunity to implement designs that meet the goals set forth in the executive order. They can make it easy for people to find, access and use the data. They can publish not just data but APIs that afford searching, filtering and exploring the data in a meaningful and helpful manner; APIs that empower both users and developers to successfully interact with the data, without resorting to a dashboard featuring dozens of options or mind-numbing explanations.

In the (likely) event that the initial open data release consists of mere data, companies and individuals would be well advised to resist the temptation to build a multitude of “one-off” applications, each of which solves a single problem or answers a narrow set of questions for some subset of the data. Instead, work should be put into converting the raw data into usable API formats such as Atom, OData, HAL, Collection+JSON and HTML (to name just a few). APIs should be designed with the same care that would be given to any interactive experience.  Investment in tools and technologies that can properly represent the data in multiple formats while supporting various use cases and access requirements will yield great results.

Open Data APIs
In the end, organizations that know the importance of a good interface, the power of choice and the freedom of flexible representations will be able to convert raw data into valuable information, which can be consumed by a wide range of users, platforms and devices. These considerations are essential to building and supporting open data APIs.

Because – ultimately – data isn’t open, unless it’s “easy to find, accessible, and usable”.

If They Have to Ask…
Try this:

• Create a new Web API
• Get it up and running on some server or other
• Hand the single URL to a client dev and say: “There ya go!”

Is the API self-descriptive? Does it contain enough information in the responses to allow client devs to know what the API is for, what it is capable of and how they can make valid requests to the server and properly parse the responses?

Here are some questions for you:

• How many assumptions do you have about your API?
• Are these assumptions shared by client devs?
• All clients devs?
• Even ones who have never met you?

If your answer to any of those questions was “No” or “I’m not sure” then it’s likely that devs will need to ask you a thing or two about how to properly use your API. That’s no big deal, right?

…You Didn’t Afford It
In everyday life, if people have to ask how to use a device (television remote, toaster etc.) then you can be sure that device is “poorly afforded” – it’s a case of weak design. We all know devices (especially electronics) that come with huge manuals and complicated explanations – and we all know what a bummer it is when that happens.

In this respect, your API is the same as any other consumer device. It should be “well afforded” – developers shouldn’t have to read the technical equivalent of War & Peace before they are able to successfully use your API.

Yes, you can supply detailed instructions in prose, provide a long list of possible methods, include lots of tables etc. These resources are helpful for devs but they can be daunting to read and cumbersome to maintain.

Another approach is to include this kind of information in a machine-readable format – and one that most devs will also understand quickly. This can be achieved by providing instructions (that get automatically updated whenever your API changes) via hypermedia controls in the response. Why write a Web page of documentation to tell devs to construct a URI and use that URI to execute an HTTP GET when you can just include that (and much more) information in your API responses?

Help your client devs out. Throw ‘em a bone, here. Don’t make them read pages of documentation when you can just include simple run-time instructions as they’re needed.

In conclusion: If they have to ask, you didn’t afford it.

(Originally published on my personal blog.)

Layer 7’s March 14 webinar, ROI for APIs – which will feature input from TechCrunch and AT&T – should help to clear up some of this confusion. In particular, the webinar will focus on how hackthons can be used to gather valuable data for API ROI measurement.

How you measure your API ROI will depend on the purpose your APIs play in the greater business picture. Therefore, to provide a little primer for the webinar, I thought it would be helpful to give examples of a few API business models and how they might generate revenue.

• Per API Call
  Text messages sent via an API are billed at $0.01 per message
• Per API Payload
  Voice transcriptions via an API are billed at $0.01 per word
• Transactional Revenue
  An API call delivers a purchase
• Firehose API
  A monthly subscription provides unlimited API access
• Platform API
  An existing SaaS platform provides an API for partner integrations

To learn more, register for the webinar – ROI for APIs: Using Hackathons to Evaluate Your API Program featuring TechCrunch and AT&T.

For example, hackathons provide a fantastic way to grow developer awareness of your API as a brand in and of itself, separate from your core business. When the developers who attend your hackathon go back to their day jobs on Monday, they have added your API to their programming tool belts and will use it, when appropriate, in upcoming projects. Additionally, hackathons will attract the attention of thought leaders and influencers who will mention your API on blogs and forums, spreading the word further. These benefits can deliver considerable value but they can also be difficult to quickly quantify.

Nevertheless, API evangelists will be held accountable for demonstrating the real-world value of their hackathons. One way to do this is to show how hackathons enable your company to conduct developer user experience (DevUX) research at a minimal cost. Gathering feedback and data from hackathons provides the most cost-effective way to optimize the quality of your API as a product by answering questions like:

• How user-friendly is my registration process?
• Do my APIs ever return incorrect or unexpected results?
• What new features should I add to future versions of my API?
• Is the skill level of my API appropriate for long-tail app developers?
• What kind of tutorials and other documentation will my developers need?
• Which programming languages are my developers using to implement my APIs?
• How useful is my API and what are the most common/innovative use cases for it?

The data and feedback you gather will also help you to further demonstrate ROI by providing the answers to questions such as:

• How many developers registered and how many actually attended?
• Did the hackathon appeal to the types of developer we want to attract?
• Did any valuable or innovative apps get prototyped?

Hackathons offer a fantastic way to build excitement around your API and optimize the quality of your interface. If you still have any doubts, join us for a hackathon (and participate!) to see how other API platforms are doing it.

At the Layer 7 API Academy, we’ve had a few requests from API designers who are seeking strategies for handling large amounts of data in API responses.  Pagination is the most common method for addressing this scenario. Pagination, which is very common on the Web, allows API architects to conserve resources, improve response times and optimize the user experience. It’s a way of splitting up data into “pages” and is used in just about any API that returns collections of data.

I’ve released a short video tutorial titled Use Pagination in Web API Design to introduce the ins and outs of the interface. This video provides a crash course explaining pagination and outlining how to use it effectively in the design of Web APIs. I couldn’t fit all the implementation considerations I wanted in this six-minute tutorial, so watch out for a follow-up video on the subject.

Each technology, service or coding framework exhibits its own “style” for solving a problem. Sometimes we select a system component because it’s familiar (“We use SQL databases because that’s what we’ve always used”). Sometimes we include one because it’s unfamiliar (“We’ve never used Node.js before, let’s try it on this project”). And sometimes we select components based on skill set (“Our team doesn’t have any experience with WebSockets, so let’s just use HTTP instead”). It’s important to step back and get a big picture view when selecting components for a production system that will (hopefully) serve your needs for an extended period of time. And that’s where architectural styles come into play.

Architectural styles set the tone for how components in a system interact, govern the implementation details and establish lines of responsibility and maintenance over time. Setting the style early on and communicating it to the team ahead of time goes a long way toward creating a stable and successful implementation. To help clients get a handle on this topic, I commonly identify three widely varying-styles for Web solutions that people can easily recognize: Tunneling, Objects and Hypermedia.

The Tunneling style is best illustrated by SOAP-based implementations where all requests are “tunneled” through a limited set of components (user management, product services etc.) exposed on the Web. The Object style is one that uses the HTTP CRUD pattern (create-read-update-delete) where domain objects (users, products etc.) are exposed and basic read/write operations are supported for those objects. The Hypermedia style relies on a shared understanding through a message format (media type) that defines both the data elements (users, products etc.) and the possible actions (read, write, filter, report etc.) on those data elements. Each of these styles can be used to implement a solution and each of them has associated benefits and challenges.

This comes up so often that we’ve created a short API Academy video introducing the subject of architectural styles for the Web. Take a look and see if it gives you some ideas for how you can answer this question the next time you are about to embark on a major system implementation: “What architectural style is appropriate for this Web solution?”

That’s why we at the API Academy stress the importance of making rational decisions rather than irrationally selecting design patterns based on emotion or trends.  We want you to choose your design elements rather than picking them from the latest set of formats or technologies that you’ve heard about.

And that’s why we’re working on a series of tutorial videos, as my colleague Mike Amundsen recently announced. The first of these videos, titled The API Interaction Model – An Introduction, provides an overview of  a design process that will help you consider your user’s perspective in order to make effective design choices later. The ideas I discuss in this video are rooted in user-centered design processes that have been very effective in the software and product design worlds.

If you’re currently designing an API, invest five minutes and watch the video.  It should be time well spent.