June 9th, 2014

Jailbreak Your APIs

Jailbreak Your APIsAt this week’s Computers, Freedom & Privacy Conference, I gave a presentation titled Jailbreak Your APIs, in which I explored the concept of “linked APIs” and explained the potential these interfaces have for helping us create a freer, more open world. The global informational overload that we are constantly exposed to, can be overwhelming but ideas like linked APIs help us remember that the explosive surge of available data also brings us beautiful things such as transparency, openness and an unprecedented feeling of global connectedness.

We’ve never felt more connected to the rest of the world than we do now. Computers, mobile devices and the Internet have brought us closer than ever before. We now take it for granted that a person can be pretty much anywhere in the world and still get a real-time, front-row view of breaking news from half-way around the globe. While this disappearance of informational boundaries has surfaced many of our most polarizing differences, we still cherish our unprecedented ability to access information because access to information has always been our most powerful weapon for defending our rights and liberties.

In this context, the White House’s Open Data Policy (part of the Open Government Initiative) is particularly exciting. Never before has the American public had so much access to government information, at all levels. And all this happens directly through the Internet, in near real-time. The ability to access this information in a timely manner is crucial – we need access to information right when it is immediately relevant to guaranteeing our freedoms. This relates to my work with CA Layer 7’s API Academy because APIs supply the core technology for facilitating timely access to data.

Recent growth in the prominence of APIs is not simply a reaction to Open Government and Open Data. The API has organically become more important in recent years, due to our increasingly mobile lifestyles. APIs are vital to mobility because they connect our mobile devices to the cloud – specifically, to the datacenters that host the information and functionality that powers our apps. APIs have played an undeniably critical role in the mobile revolution of recent years. However, for APIs to play a similar role in the Open Data revolution, we need them to become much better.

The problem with APIs right now is that most of them are, at best, creating narrow windows into solid walls surrounding siloed data. Even the biggest, most well-known APIs (such as those provided by Twitter, Facebook and Google) to a large extent, only operate on the data that is within these organizations’ own databases. And most Government APIs don’t even allow any “write” functionality – they are strictly read-only.

In that sense, most current APIs create isolated, guarded data islands. This is very “anti-Web” — the World Wide Web was created in the spirit of decentralized equal participation. On the Web, everybody publishes everywhere, owns their data and then we have ways to reach that data through hyperlinks, through Google search and other methods. APIs have not really reached that stage of maturity yet. APIs are highly centralized, in terms of data storage and virtually none of them ever link to other APIs.

We need a new breed of interfaces: linked APIs, based on the same hypermedia design that we have on the rest of the Web. Such APIs will have the biggest impact for Open Data because they will link and make connections across datasets and organizational boundaries. Linked APIs are also very scalable, so they will be best suited to meeting the challenges of Big Data. After all, the Web is the largest, most distributed network of information humankind has ever created. We know the architecture of the Web can scale and linked APIs have the exact same architecture, with hypermedia as the engine.

For freedom of data, we really need more linked APIs. We can only truly have open and free data if we jailbreak the information out of the silos it is currently stashed-away in. Linked APIs provide us with keys to the data fortresses where large aggregators currently keep data. Linked APIs can ensure that our data isn’t stashed in centralized warehouses. Linked APIs represent the engine of data freedom on the Web. Let’s get the engine cranking!

May 29th, 2014

Toward a Lean API Strategy

Books on Lean Business Strategy“Lean”, “API” and “IoT” are probably the most hyped terms in our industry right now. Normally, I tend to blog about the latter two but – for a change – I would like to balance that out by talking about the former: the concept of Lean and how it relates APIs.

You have probably heard about or even read The Lean Startup by Eric Ries. And you may have noticed that this book sparked a whole cottage industry of Lean publications, like Lean Analytics, Lean UX and the widely-misunderstood concept of “minimal viable product”.

But few of you may have ventured back to explore the texts that laid the foundation for the Lean startup. The Four Steps to the Epiphany by Steve Blank, for example. Blank outlines a business process called “customer development”, which helps startups find “problem solution” and “product market” fit. Even fewer will have ventured right back to the very origins of the Lean concept: the Toyota Lean Production System with its emphasis on pull over push and ever-decreasing batch size towards one-piece-flow manufacturing. And we have not even touched upon the “theory of disruptive innovation” that Clayton Christensen outlines in The Innovator’s Dilemma or Rita McGrath’s concept of “discovery-driven planning” outlined in Discovery Driven Growth.

But the purpose of this post is not to provide a comprehensive reading list for those of you hoping to learn more about Lean and discovery-driven business strategies. My real goal is to explore if and how these concepts can be applied to API design best practices. However, if you are curious and want to know more about the books mentioned, I suggest you head over to a blog post I wrote for launchd.io. And for your next long-haul flight, you might want to consider starting with The Goal, which will provide you with some truly novel-like business reading.

Before I explore how Lean and API design come together, let me first make a confession – I got an MBA a couple of years back. I know that this is not going to win me any brownie points and I still prefer code to spreadsheets – no contest! But it goes some way to explaining why I think business and API are joined at the hip. The business value of an API does not come from the interface’s intrinsic technical features but from its ability to provide access to a business asset or service. APIs provide a technical means to do (more) business.

From this follows the assumption that API design and implementation need to focus on the intended business outcome. Which means that you must have a clear view of your business goals before you can start to implement your strategy. Unfortunately – in my experience – most of us on the technical side are not equipped to talk to the business side of the house (and they are seldom well prepared to talk with us). This is why I went back and got an MBA – so that I could learn to speak with “them” and build better products.

Starting with Toyota’s lean product development process, I began to see approaches and tools that could help bridge the semantic gap between the technical and business sides. I plan to share some of these with you in subsequent posts. I will start by discussing Alexander Osterwalder’s Business Model Canvas. To get a little background on what I’ll be talking about in my next post, I suggest that you read this post on ProgrammableWeb, where Mark Boyd uses the Business Model Canvas to analyze Walgreens’ QuickPrints API. Also, you might want to take a look at my Lean API Strategy presentation from the recent APIdays in Berlin.

May 23rd, 2014

Mapping the API Landscape

Mapping the API LandscapeThis week, I had the opportunity to deliver a “lightning talk” at the APIStrat Tech Un-Workshop at Gluecon 2014. The event was focused on two key topics: IoT and service description/discovery. I was in the service track and delivered a talk called Mapping the API Landscape. I won’t cover the entire talk here (BTW, the text has lots of links to information I could not discuss on stage this week) but did want to hit some key points.

What Google’s Self-Driving Car Tells Us
The Google car has been in the news again and a key point that was discussed at some length in these articles was the fact that the car depends on a very detailed map of the roadways. Right now, the car can only drive in the Mountain View, CA area since that is the only landscape mapped well enough for the car to navigate.

So, the Google car does not “discover” anything while driving. It actually recognizes intersections, traffic lights, etc. through a special representation of the landscape that contains all the right annotations. This reliance on a known map allows the car to navigate successfully between two points within that landscape – which is no simple feat, of course. Reacting to surroundings “at speed” takes serious computing power and that’s one of the reasons the Google implementation is so amazing.

Norman’s Action Lifecycle
The process of navigating from A to B is a goal-driven process that we see very often in nature. Ants, micro-organisms etc. all do this. Human-computer interaction expert Donald Norman calls this process the “action lifecycle” or “seven stages of action”.

This is how we learned to write GUI interfaces, too. Wait for a keystroke or button-click, process that action, affect the UI, then allow the user to evaluate the changes and decide if another action is needed. We also build Web servers this way. Wait for a request, process it, modify the backend (if needed) and reflect results back to the requestor. Game programming works like this, as well. But it’s rare to see the Web and mobile client applications that leverage enterprise APIs written in this way. They continue to look like single-minded bots that just go from A to B and ignore landmarks, incapable of actually reacting to surroundings.

Client Apps & Web Maps
Why are most client apps one-off implementations that are likely to break over time? It’s because client developers don’t have decent “maps” of the online landscape. And good maps are not just “photos” of the surroundings but heavily-annotated representations with recognizable symbols and landmarks. Most servers today just belch out JSON or XML with almost no recognizable symbols or signage (e.g. hypermedia controls).

So, what we need to do is create maps for devs to use so that they can build their own client applications against our APIs and solve their own problems. Client apps should be free to follow whatever route they wish within the maps – they shouldn’t be limited to following the path that server developers decide upon.

Let’s Make Maps!
Things like service description formats and discovery protocols are all ways to start creating more maps for client devs to rely upon. Using hypermedia in responses provides the symbols and signs client apps can use at runtime (like the Google car) in order to navigate in real time.

There are several description formats (see my paper Hold Your Nose vs. Follow Your Nose for more on this). In the book RESTful Web APIs, Leonard Richardson and I list close to 20 options for expressing hypermedia in Web responses. And more have come online since the book was published last year.

We have all the tools we need. We just need to use those tools to make more maps!

April 24th, 2014

SDKs Work Until They Don’t

SDKs and APIsOver the last couple of months, I’ve had many great opportunities to road test my findings on SDKs vs. APIs, with a wide variety of audiences – at our API Academy Summits in New York and London, on tour with the Nordic APIs team in Stockholm and Copenhagen and at my most recent API Workshop in Istanbul. Given the apparent relevance of the topic and the lively discussions I’ve had around it, I’d like to take this opportunity to summarize some of the insights and recommendations that have come up.

If you’ve been following our blog, you will remember that I started this topic with the observation that leveraging an API increasingly seems to involve using an SDK rather than the API itself. I followed up with a post talking about my decidedly mixed experiences of trying to use SDKs. My own experiences inspired the headline of this post: SDKs work until they don’t.

So, what are the main motivations to invest in an SDK, from an API-provider perspective?

  • Simplifying API design by extracting business logic into the client
  • Maximizing scalability by exploiting client-side processing
  • Empowering developers to leverage the API more quickly
  • Presenting an optimized client from a target-platform perspective (e.g. for mobile connectivity or constrained hardware)
  • Providing a strongly-typed presentation of the API in a variety of programming languages

Let’s contrast all this with the main drawbacks of SDKs:

  • Picking which platforms, languages and frameworks to support – some of your target developers are going to end up disappointed
  • Relying on third-party frameworks – any developer who has to integrate with two or more APIs leveraging the same framework at different version levels is bound to experience some headaches, for example
  • Adding carry-on weight of unused functionality to the application
  • Incurring long-term support costs for the SDK

But to me, the biggest risk of a SDK-first approach lies in making API design an afterthought. We have come to this point in the API evolution because pragmatic REST introduces just enough constraints to force us to think about how we can abstract the underlying business asset into a resource-based model restricted to CRUD-style interactions. An SDK-first approach might tempt us to go back to a RPC-style API design mirroring the backend implementation, resulting in all the inherent integration complexities we had with Web services.

So, if and when you decide on a SDK, keep the above in mind. You might still come to the conclusion that you need an SDK in order to quickly onboard developers or provide the best client for your API but at least you will be able to consciously weigh the benefits against the drawbacks.

If you want to dig more deeply into the subject, I can highly recommend the following podcasts, articles and blog posts, which helped greatly in forming my own opinions:

April 17th, 2014

Next API Tech Talk: Linked APIs

Linked APIsThe challenges faced by today’s software architects go far beyond the familiar. “Big Data” means more than managing petabytes of data – it requires dealing with data-sets that span organizational boundaries. Likewise, the term “distributed system” no longer refers to just a multi-tier architecture or cloud deployment – it usually involves the connection of non-heterogeneous systems across multiple organizations.

On Thursday April 24, I’ll discuss these challenges as part of Layer 7’s latest API Tech Talk. I’ll be using this opportunity to explore how architects can leverage “linked APIs” to handle Big Data sets and distributed systems that cross organizational, technological and cultural boundaries, breaking through data silos in order to better integrate information. Interested? Just add the Tech Talk to your calendar and go to api.co/L7live at 9am PDT (12pm EDT) next Thursday.

I’ll also be taking your questions on linked APIs, Big Data, distributed systems, open source and anything related, so please don’t hesitate to join in. You can submit your questions now by email or you can chat with me or tweet them at me on the day. This will be my first Tech Talk since joining the Layer 7 API Academy and I’m really, really looking forward to a lively discussion. See you on April 24!