At the Layer 7 API Academy, we’ve had a few requests from API designers who are seeking strategies for handling large amounts of data in API responses.  Pagination is the most common method for addressing this scenario. Pagination, which is very common on the Web, allows API architects to conserve resources, improve response times and optimize the user experience. It’s a way of splitting up data into “pages” and is used in just about any API that returns collections of data.

I’ve released a short video tutorial titled Use Pagination in Web API Design to introduce the ins and outs of the interface. This video provides a crash course explaining pagination and outlining how to use it effectively in the design of Web APIs. I couldn’t fit all the implementation considerations I wanted in this six-minute tutorial, so watch out for a follow-up video on the subject.

When talking to clients about the architectural details of an implementation, one of the first questions I ask is: “What architectural style is appropriate for this Web solution?” It turns out this question stumps most of my audience. Not many system architects and developers think about it. Instead, they implement solutions using whatever components and frameworks are on hand.

Each technology, service or coding framework exhibits its own “style” for solving a problem. Sometimes we select a system component because it’s familiar (“We use SQL databases because that’s what we’ve always used”). Sometimes we include one because it’s unfamiliar (“We’ve never used Node.js before, let’s try it on this project”). And sometimes we select components based on skill set (“Our team doesn’t have any experience with WebSockets, so let’s just use HTTP instead”). It’s important to step back and get a big picture view when selecting components for a production system that will (hopefully) serve your needs for an extended period of time. And that’s where architectural styles come into play.

Architectural styles set the tone for how components in a system interact, govern the implementation details and establish lines of responsibility and maintenance over time. Setting the style early on and communicating it to the team ahead of time goes a long way toward creating a stable and successful implementation. To help clients get a handle on this topic, I commonly identify three widely varying-styles for Web solutions that people can easily recognize: Tunneling, Objects and Hypermedia.

The Tunneling style is best illustrated by SOAP-based implementations where all requests are “tunneled” through a limited set of components (user management, product services etc.) exposed on the Web. The Object style is one that uses the HTTP CRUD pattern (create-read-update-delete) where domain objects (users, products etc.) are exposed and basic read/write operations are supported for those objects. The Hypermedia style relies on a shared understanding through a message format (media type) that defines both the data elements (users, products etc.) and the possible actions (read, write, filter, report etc.) on those data elements. Each of these styles can be used to implement a solution and each of them has associated benefits and challenges.

This comes up so often that we’ve created a short API Academy video introducing the subject of architectural styles for the Web. Take a look and see if it gives you some ideas for how you can answer this question the next time you are about to embark on a major system implementation: “What architectural style is appropriate for this Web solution?”

API design can be daunting. With so many decisions to make and so many differing opinions available on interface design, it’s easy to feel frustrated by the process.  Even worse, it’s possible to follow bad advice and end up designing an API that developers hate using.

That’s why we at the API Academy stress the importance of making rational decisions rather than irrationally selecting design patterns based on emotion or trends.  We want you to choose your design elements rather than picking them from the latest set of formats or technologies that you’ve heard about.

And that’s why we’re working on a series of tutorial videos, as my colleague Mike Amundsen recently announced. The first of these videos, titled The API Interaction Model – An Introduction, provides an overview of  a design process that will help you consider your user’s perspective in order to make effective design choices later. The ideas I discuss in this video are rooted in user-centered design processes that have been very effective in the software and product design worlds.

If you’re currently designing an API, invest five minutes and watch the video.  It should be time well spent.

I’m happy to announce the release of the first API Academy video shorts. I’ve been working with my colleague Ronnie Mitra to create a series of short (five-minute), informative videos on topics related to the Web, APIs and solution design/implementation.

These first few videos are just the start. We plan on doing more of these shorts on a wide range of topics, over the coming weeks and months. And we need your help. Please take a look at these first vids and send us your feedback.

You can comment here, on YouTube or by emailing me directly. We’re looking for feedback on the format, suggested topics and even how we could improve upon this model (hosting a separate site, adding interaction, badges etc.)

Any time you can spend on watching these and sending comments will be most appreciated. Our aim is to do something helpful, engaging and – above all – enjoyable. Thanks for your help and let’s see what this can become!

The API Interaction Model – An Introduction

—

Three Common Web Architecture Styles

—

Handle Errors on the Web

Attend any Web API presentation and you are likely to see a graph like this one, demonstrating the growth of  publicly-available Web APIs. Speakers (including me) love using these graphs for good reason: they succinctly capture the explosive growth of APIs that has taken place over the last two years.

It’s a great story but it’s really only half the story. Web API experts regularly acknowledge the existence of a “private” or “closed” API market. In fact, many of us believe that if the number of private APIs in use could be cataloged it would dwarf the 7,000 or so APIs that are published on the ProgrammableWeb site.

As with many of the terms in the API world, there isn’t a concrete definition of  “private API”. In general, a private API has these characteristics:

1. It provides a language-independent interface that is made available via Web protocols.
2. It’s access is limited to a specific set of known developers or organizations.
3. It is not marketed to the general public nor is its documentation made publicly available.

Further to this, we can divide private APIs into three major buckets:

1. Internal APIs that exist within the organization’s borders (for example, SOAP-based interfaces within an internal Service Oriented Architecture).
2. Business-to-business (B2B) APIs that enable organizations to integrate with external companies.
3. Backend APIs that drive mobile, Web and device-based applications.

With this definition we can see that there are a great many integrations that must already exist. Enterprises have been building SOAP and B2B-based connectivity for years and have accumulated healthy inventories of private APIs.

In addition, the headlong rush towards the world of mobile is driving the creation of new externally-facing APIs to help corporations reach their customers. As I’ve talked about before, many organizations wish to retain control over the development of these applications and they can do this with private APIs.

If IT teams have been building these types of in-house connectivity solutions for so many years, there shouldn’t be much room left for innovation or improvement, right?

Not quite. Unlike those who build private APIs, public API designers are motivated by the need for their interfaces to be chosen out of the mass of APIs that are available to their prospective users.  This difference in motivation has created a massive impact on how public APIs are designed and managed. Architects responsible for private APIs have a great opportunity to learn from the public API world by adopting design strategies devised to drive adoption, in a controlled manner.

A good reason to take a developer-centered approach to private API design is the development cost associated with building applications that utilize the interface.  A well-designed private API can reduce the project costs for application development as well as for maintenance and upkeep of the integration.  Good design isn’t easy but it pays off – even when the audience is limited.

Many enterprises are implementing a “private for now and public later” API strategy.  It is a great idea but that doesn’t mean architects shouldn’t strive to incorporate great API design and a solid management solution.

In my next post, I’ll dive into private APIs in more detail and talk about some of the specific challenges that arise when building closed interfaces and how these challenges can be addressed with management solutions.