Mike Amundsen

Mike Amundsen

Mike Amundsen is Layer 7’s Principal API Architect. An internationally-known author and lecturer, Mike travels throughout the United States and Europe, consulting and speaking on a wide range of topics including distributed network architecture, Web application development and cloud computing. His recent work focuses on the role hypermedia plays in creating and maintaining applications that can successfully evolve over time. He has more than a dozen books to his credit, the most recent of which is Building Hypermedia APIs with HTML5 & Node. He is currently working on a new book on “cloud stack” programming. When he is not working, Mike enjoys spending time with his family in Kentucky.

October 7th, 2013

SDKs or APIs: What’s the Right Choice for Your Developer Community?

SDK or APIWhen creating an application programming interface (API) for a service, one of the key decisions any program or product manager will face is how best to meet the needs of their prime target audience: developers. Faced with this decision, you want to make sure your API is easy to use and doesn’t represent a high barrier to entry for your specific developer audience.

Currently, the typical approach is to design an interface that leverages the most common protocol on the Web today: HTTP. This is often labeled a “RESTful” API (referring to Roy Fielding’s architectural model for the World Wide Web) and offered as a one-size-fits-all (OSFA) model for developers to use when building client applications for a service. But this is not always the best approach.

Properly understanding and implementing a raw HTTP interface may be too complex for some segments of your developer community – some of whom are really only interested in your service and not in spending time to build a killer HTTP application for that service. Additionally, a generic HTTP API might end up compromising key performance aspects of your service in order to work best for a wide range of developer communities (the OSFA problem). Even worse, an HTTP-based API may – in the end – result in unfocused client applications built by developers who know more about HTTP than your service.

What We Learned from SOAP
One of the powerful lessons learned from the SOAP community relates to the value of developer tooling. The SOAP Web Service Definition Language (WSDL) is a complex, difficult-to-read document that contains all the important details on building a compliant client application for a service. And developers have a hard time making sense of the document. To solve this problem, the SOAP community helped promote an “accommodation” for developers –  the “WSDL” button that is available on many code editors. By simply pressing this button and following a few prompts, developers can easily create API facades for servers or consume WSDL documents to build client applications. The WSDL accommodation makes SOAP programming not only easy, it adds to the usability of SOAP interfaces and lowers the bar for developers wanting to use services on the Web.

What We Lost with HTTP CRUD
The rise of JSON-style HTTP CRUD (Create-Read-Update-Delete) programming interfaces meant the loss of the WSDL accommodation. Developers were expected to hand-craft both the server and client interfaces without the aid of a unifying definition document like SOAP’s WSDL. To make up for this loss, server coding environments like Ruby on Rails introduced helper functions (e.g. “rails new”) designed to generate a great deal of the API facade required in order to publish a service on the Web. Developers could use this accommodation to simplify the interface details and allow them to focus on crafting the internal object and business modeling needed to make the service operational.

Client devs needed to create their own accommodations, too. That’s why we have client-side libraries like ember.js, backbone.js, angular.js and others. Like the developers building servers, client developers needed help handling the basic plumbing for HTTP-based applications, so they could focus on their own object models and business logic.

The bad news is, in this HTTP CRUD world, each and every service has its own unique set of URLs, objects (users, products etc.) and actions (approve, remove, edit, apply etc.). And each API looks like a snowflake among hundreds of other unique interfaces. For example, there are more than 500 unique APIs for supporting shopping services. This can raise the bar for developers and lower the usability of HTTP CRUD APIs.

Adapter APIs
Netflix set out to solve the OSFA problem in 2012 by embracing differences in its developer community and creating a set of targeted “adapter APIs”. These are custom interfaces optimized for selected developer communities. For example, Netflix offers one API for its XBox community and a slightly different API for its PS2 community. In fact, each major device has its own custom API.

What’s interesting about the Netflix approach is that the customized interface accommodations live on the server, not the client. In other words, Netflix has taken on the task of optimizing its interfaces for each community and hosting that optimization on its own servers. A client developer will still be using an HTTP API but it will be one tailored to a specific device – a server-side custom library.

Do SDKs Provide the Answer?
Another way to provide solid accommodations to your HTTP developers is to create software developer kits (SDKs) for your service. Essentially, SDKs provide the same accommodation that WSDL-generated client code does for SOAP interfaces. The good news is that well-designed and executed SDKs can lower the bar for developers and increase service usability.

Recently, Evernote announced that it was taking the SDK approach. One reason for this was Evernote’s decision to use the Apache Thrift message model. The Thrift approach serializes messages using a custom binary format and then ships them across the network using one of a handful of transport protocols (including direct TCP/IP). This is pretty low-level stuff and can easily intimidate some client developers – raising the barrier to entry for Evernote’s API – and this is where creating an SDK is a handy approach. Evernote has committed to building a wide range of language-specific SDKs that clients can download, install and use in their own code. This accommodation lowers the bar when using the Thrift model.

The bad news is that creating SDKs is often a recurring additional expense for your services team. Unlike the WSDL standard, which makes it possible to generate code for a wide range of programming languages from a single published definition file, SDKs usually need to be hand-built for each target programming environment. And selecting target programming languages can turn into a slippery slope. Which languages are most used by your current API consumers? What if the most-used language represents only 30% of your target audience? How many SDKs do you need to build and maintain before you reach a significant portion of your developer community?

The maintenance of SDKs can be substantial, too. How often do you release updates? Fix bugs? Provide new features? Each release can mean added cost to your developer community as they need to open up their own code, integrate the new SDK, run tests and finally re-release their apps to their own target communities. Unless carefully done, this can result in added churn and expense all around – including deployment and download costs for end users. And that can raise the barrier to entry and lower usability.

So What’s a PM to do?
When you start to think about the notion of creating accommodations for target audiences, you have a new metric for assessing the usability and value of your service interface. If you can design an API that is easy for your target audience to use, then you (and your developers) win. If your API is too complex for your audience or relies on a less-used technology, you likely need to include more direct accommodations for your developers.

In some cases, the audience will know how to use SOAP interfaces and will benefit from you offering a WSDL as their accommodation. In other cases, the target audience won’t want/need a SOAP interface and a well-crafted HTTP API (or possibly a set of targeted adapter APIs) will be the right choice. Finally, some target developers will want/need an SDK to handle the protocol details and allow them to focus on their own business logic.

In fact, sometimes the best bet is to offer more than just one of these options in order to reach all your target audiences. Your enterprise partners may prefer a SOAP interface, your mobile devs may prefer a well-design HTTP CRUD API and your business and marketing team may prefer an SDK that exposes only the parts of your service they need to use.

You Own the Interface
The key point in all this is that you own the interface. You can cast your service in many different ways, aimed at several different audiences. You don’t need to stick with a OSFA approach and you don’t have to rule out major technology sectors like SOAP just because a portion of your audience prefers one interface style over another.

By focusing on your target audience and learning their skills and preferences, you can identify key metrics that can drive the selection of the right mix of API styles and accommodations that will help you meet your goals for API reach and usability.

August 16th, 2013

Designing Web APIs – A Candid Conversation

API Design WebinarIt was just over a year ago that we hosted our first API Workshop (for the record, it was July 2012 in Sydney Australia). Since then, I and my API Academy buddies Ronnie Mitra and Alex Gaber have had the privilege to meet and talk with hundreds of developers representing dozens of companies and organizations all over the world. It has been a very rewarding experience.

Along the way, we’ve learned a great deal, too. We’ve heard about creative ways people are leveraging the Web to build powerful APIs. We’ve seen great examples of real-world APIs and learned the practices and pitfalls encountered while maintaining and growing these APIs over time. We’ve even had the opportunity to observe and participate in the process of designing and architecting systems in order to foster creative innovation and long-term stability for the APIs.

In the past year, we’ve collected many examples of best practices and distilled common advice from a range of sources. We’ve also created free API events, conducted dozens of hackathons, webinars, one-day workshops and multi-day API boot camps as ways to share what we’ve learned and help others build upon that advice when creating their own Web APIs. And at every event along the way, we’ve met more innovative people doing great things in the Web API space.

As a way to look back and compare notes, Ronnie and I will be hosting a webinar (Designing Web APIs – A Candid Conversation) on August 22 at 9AM PDT. We’ll look back at what we’ve seen on our travels and talk candidly about such topics as SOAP, SOA, REST, lifecycle management and more. It’s going to be a fun hour of both reminiscing and looking forward to this fall’s workshop series and the future of APIs in general.

Also this August, we’re taking a break from offering public events and using the time to compare notes, assess the advice and examples we’ve gathered and improve our content for the upcoming fall season. Ronnie, Alex and I (and many others here) will be spending many hours this month creating new guidance documents, articles and presentations/videos – all in the effort to share what we’ve learned and help others make a difference within their own organizations.

I hope you’ll join us on August 22 for our Webinar and I hope you’ll keep an eye on our workshop schedule for upcoming events near you. Even if you’ve participated in our open workshops before, you’ll want to come back for the new series. We’re adding new topics, brushing up existing material with new guidance from the field and adding new features to the events.

August 9th, 2013

REST Fest 2013 is Coming!

REST Fest 2013It’s that time of year again! REST Fest 2013 is less than two months away (September 19-21) and preparations and are in full swing. Now in its fourth year, REST Fest has become one of my favorite events on the calendar and I’m very much looking forward to being involved with this year’s event.

REST is Just the Beginning
This year the keynote will be delivered by Brian Sletten. And – judging from the title (and my knowledge of Brian’s experience and knowledge) – it will be a great talk. We’re honored that Brian accepted our invitation and looking forward not just to his presentation but also the resulting converstations and explorations that are hallmarks of REST Fest.

Everybody Talks
An important part of REST Fest is the principle that everyone who shows up must give a presentation. The talks are typically quite short: a five-minute “lightning” talk followed by a short Q&A session. There are a few 30-minute  “featured talks”, too. But the basic idea is that we all get to talk about things that are interesting to us and we don’t have to make a big deal about it.

Every year, I probably learn more than 30 new ideas and novel approaches to problem solving and get to talk to the people who are coming up with these great things. REST Fest is a fantastic boost to my creative spirit!

Everybody Listens
The corollary to our key “talk” principle is that we all get to listen, too. And listening is, in my opinion, even more important than speaking. REST Fest attendees come from all sorts of backgrounds, experiences and points of view. The chance to hear how others view the Web space, how others are tackling problems and how others are advancing the practice of services on the Web is always an eye opener.

Less Theory, More Practice
And that leads to another key aspect of the weekend. The focus is on doing, not theorizing. We’re a decidely non-pedantic bunch and are usually much more interested in cool solutions than compelling theories. While it may still be common to think of anything with the REST acronym in the name to be a meeting of pointy-headed geeks, that’s not us. Each year, I get to see actual code solving actual problems in the real world.

We Hack, Too
Every year, we also host a hack day where everyone gets together to work on cool REST-related Web stuff. This year, Erik Mogensen will be leading the day. From what I’ve seen, he’s got some cool ideas in store for us, too.

It’s Easy to Join Us
Just as we cut down on the ceremony surrounding speaking and participating in a conference, we also try to eliminate the ceremony around signing up and showing up for REST Fest. It’s quite easy:

  1. Join our mailing list to see what we’re all about
  2. Drop into the IRC channel to chat us up
  3. Hop onto the GitHub wiki and create your “people page”
  4. Head over to the registration page and reserve your seat for the event

There’s no waiting to see if your talk was accepted; no wondering if what you’re working on would be interesting to some review committee. Just sign up, post your ideas and head down to sunny Greenville, SC for a great weekend.

Need More REST Fest NOW?
Can’t wait for RESTFEst 2013 to get started? Take a look at our Vimeo channel with all the talks from previous years. There’s lots of very cool stuff there.

See you in September!

(Originally published on my personal blog.)

July 23rd, 2013

Interoperability, Not Integration

Interoperability Not IntegrationIt’s a small semantic difference, really but a difference I think is worth calling out. When working in large distributed systems, it’s better to aim for interoperability than integration. And here’s why…

Integration for a Single System
Part of the Merriam-Webster Online Dictionary’s definition of integration is particularly relevant here:

A common approach to working with large distributed systems – e.g. internal networked implementations that run at various locations within a single organization or implementations that rely on some Web-based service(s) – is to attempt to treat the entire operation as a single unit, a “whole system”.

Bad idea!

These “whole systems” can also be called “closed systems”. In other words, people work to create a fully-controlled single unit for which, even when elements are separated by space (location) and time (“We built that part three years ago!”), there is an expectation that things will work as if they are all physically local (on a single machine) and temporally local (there is no significant delay in the completion of requests). As you might expect, attempting this almost always goes badly – at least at any significant scale.

There are several reasons for attempting this approach. The most common is that treating everything as “your system” is mentally easy. Another reason this single-system view prevails is that most tooling acts this way. The legacy of edit and build tools is that all components and data are local and easily accessible. How else would we be able to do things like code completion and data model validation?

Anyway, the point here is that “integration” is an anti-pattern on the Web. It’s not a good idea to use it as your mental model when designing, implementing and deploying large-scale systems.

Interoperability for Working with Other Systems
As you might have guessed, I find Merriam-Webster’s definition for interoperability much more valuable:

The interoperability mindset takes a different approach. In this view, you want – whenever possible – to treat things as interchangeable; as things that can be swapped out or re-purposed along the way. Interestingly, Merriam-Webster notes the first known use of this term was in 1977. So, the idea of interoperability is relatively new compared with “integration”, which was first used in 1620, according to Merriam.

An interoperability-focused approach leads to systems that do not need to “understand” each other, just ones that use interchangeable parts. Especially in widely-distributed systems, this interchangeability has a very high value. It’s easier to replace existing items in a system (e.g. changing data-storage vendors), re-use existing parts for other needs (e.g. applying the same editing component built for a blogging service to a new print publishing service) and even re-purpose parts when needed (e.g. using the file-based document caching system to provide caching for logged-in user sessions).

The primary challenge to thinking like an inter-operator instead of an integrator is that there are no easy tools for this kind of work. Pretty much all integration work is done by creative thinkers in the field (“We could just use our existing storage system for that.”) You usually need a rather specific knowledge of what’s available on site and what the existing parts can do in order to execute on interoperability.

Despite the extra cost of interoperability, there are important benefits for distributed systems that must operate over a long period of time. That’s why so much of the Web relies on interoperability. The standards we use for DNS, HTTP, HTML etc. all assume that varying products and services are free to decide what they do and how they do it as long as they inter-operate with other products and services on the Web.

Treat the Network Space as a Bunch of Other Systems
If you take the approach of treating everything in your network space (e.g. your local intranet or any system that relies on at least one Web-based service) as a bunch of “other systems” you’ll be better off in the long term. You’ll stop trying to get everyone to work the same way (e.g. using the same storage model or object model or resource model) and will be free to start working with other teams on how you can share information successfully across systems, via interoperability.

Even better, large organizations can get a big value out of using the interoperability model for their implementations. In practice, this means fostering an internal ethos where it’s fine to be creative and solve problems in novel ways using whatever means are at your disposal as long as you make sure that you also support interoperability with the rest of the parts of the system. In other words, you have the freedom to build whatever is most effective locally as long as it does not threaten your interoperability with the other parts.

There are lots of other benefits to adopting interoperability as the long-term implementation goal but I’ll stop here for how and just say, to sum up:

  • As a general rule, treat your implementations as exercises in interoperability, not integration.

(Originally published on my personal blog)

July 10th, 2013

Chicago, Sydney, Melbourne, Toronto

Layer 7 API WorkshopsOver the span of about two weeks, I’ll be visiting four cities, three countries and two continents, as part of Layer 7′s continuing free API Workshop series. Along the way, I’ll be joined in each city by great folks from both Layer 7 and CA Technologies.

Layer 7 has already hosted lots of How to Implement a Successful API Strategy workshops this year, across Europe and North America, with content delivered by my API Academy colleagues Ronnie Mitra, Alex Gaber, Holger Reinhardt and Matt McLarty. Over the last few months, I’ve had the pleasure to meet dozens of attendees working on some incredibly interesting projects using APIs on the Web and on internal networks.

Each half-day event includes high-level summaries of the most popular topics from our Introduction to APIs Workshop and API Design & Architecture Boot Camp and – like all our workshops – each is highly interactive. Whether you are just starting to consider incorporating APIs into your distribution model or are already well into a live implementation, these sessions provide a great way to see and hear how others are approaching the same space and to ask questions about how you and your organization can improve the design, implementation and lifecycle maintenance of your Web-based APIs.

Here’s where I’ll be during the next two weeks:

  • Chicago – Jul 16
    If you’re in the US Midwest, there are still a few open seats for this workshop.
    Register now >>
  • Sydney – Jul 24, Melbourne – Jul 25
    I’ll be joined at the Sydney and Melbourne events by Layer7′s CTO Scott Morrison.
    Register for Sydney >>
    Register for Melbourne >>
  • Toronto – Aug 1
    This one will include a presentation from Layer 7 co-founder Dimitri Sirota.
    Register now >>

We’re getting great feedback from attendees, so if you haven’t been able to attend one of our workshops yet this year, now is a great time to pick a location near you, sign up and see what the fuss is all about. One more thing: If you don’t see a convenient location on the list, don’t worry. We’re already gearing up for our fall schedule and you’ll be seeing lots of new locations and content appearing soon.