July 15th, 2014

Beyond the CMS

NPR BuildingOn April 22, 2011, I was in Washington, DC, preparing to start my new job at NPR. At that point in my life, this was pretty much my dream job, so I was very excited and a little nervous. I did a lot of thinking that night and the conclusions I came to eventually became the basis of NPR’s technology strategy. I recently had a chance to share my thoughts from that night as part of a talk at the Integrated Media Association’s iMA 2014 conference. Here are the edited highlights.

The basic premise I started from was that all content management systems are fundamentally broken. This may sound a little harsh but I feel able to say it because I’m part of the problem – I’ve built content management systems for organizations across the public and private sectors, so I’m pretty well placed to tell you that no available CMS platform is architected for what publishers – particularly news outlets – truly need.

Most content management systems were designed years ago, for a much simpler world. We now live in an incredibly fragmented and complex world. Any piece of content tends to be sourced from a variety of places and published across a range of old and new media channels. Throughout this complex process, everything has to work seamlessly. The margin for error during breaking news or major events is pretty much zero.

In this context, what do publishers actually need from a CMS? They need:

  • An easy way to connect with many news sources
  • The ability to push content across a variety of channels
  • Guaranteed availability and scalability

So, how do we build a CMS that actually addresses these needs? To my mind, the solution has three key components. First and foremost, the whole architectural approach must be based on APIs. Second, it must specifically use hypermedia APIs and finally, the APIs must be what I’ve been calling “linked APIs”.

1. APIs First
APIs represent the only universal way to connect anything on the Web to any other online thing. Unfortunately, since we started the Web in a desktop-centric world, APIs were an afterthought. Historically, we used to build a Web site and then maybe also add an API, as a window into our content.

This is the wrong approach. Your Web site is just one of the destinations for your content. Increasingly, it’s not even the most important one, since mobile viewership is clearly on the rise. Don’t treat your Web site as special. All your content and functionality should be put into and delivered through APIs.

 2. Hypermedia
Publishers need things to just work. They don’t care about the technical details; they just can’t have their services go down at any time – so, scalability is paramount. And how do you ensure scalability? As I’ve pointed out before, the most scalable network ever created is the World Wide Web and the secret to the Web’s scalability is hypermedia.

Hypermedia is any type of content that not only carries data but also links to other documents. The hypermedia type that is most fundamental to the Web – and certainly the one we are most familiar with – is HTML. However, HTML was designed for human-centric Web sites, not for exchanging structured content via APIs.

There are, however, other hypermedia types that were designed for this very purpose. As a matter of fact, I was involved in the creation of a very robust one called Collection.Document, which was designed specifically for media organizations.

3. Linked APIs
Leveraging hypermedia as an integral part of interface design allows us to create “linked APIs”. Most current APIs are, at best, creating narrow windows into the solid walls of data silos. Even the most high-profile API will typically only provide access to a single corporate database. Hypermedia allows us to create links between these databases.

This will prove essential to the next generation of content management systems because linked APIs have the potential to give content publishers the freedom they want to seamlessly integrate content from diverse sources and push it across the full spectrum of online channels. As such, they could even come to represent the engine that drives press freedom into the coming decades. So, let’s get that engine cranking!

June 26th, 2014

APIs in the Connected Car: APIdays San Francisco

APIdays SFToday, I’m going to share some rather opinionated thoughts about APIs and the connected car. My opinions on this subject sprang from a combination of real-world experience plus (informed) speculation and came together as I prepared a talk for APIdays San Francisco.

The connected car is widely recognized as a game changer for the automotive industry. Experts all agree that just selling cars is a thing of the past. Mobility, connectivity and in-car user-experience will be leading decision considerations for car sales. Right now, automotive manufacturers, content providers and app developers are all competing to take a leading role in the connected car space. This is a matter of survival. Winners of the competition will be richly rewarded; the losers may sink into oblivion.

Car manufacturers seem understandably determined to dominate the connected car space. But this space is inherently shared with device manufacturers, content providers and app developers. Take away any one participant and you no longer have a sustainable ecosystem. If the automotive sector is not prepared to work with and accommodate the needs of other stakeholders, then no one will win. There are three things the industry can do to make things significantly better right away.

1. Implement a Standard Hypermedia Type for Automotive APIs
Right now, every car manufacturer wants to do its own thing and sees originality as a key to differentiation. This is a fallacy. There are way too many car manufacturers for content providers and app developers to keep up with the variety. Some have suggested that all manufacturers should just deploy Android as the base OS. I personally doubt they will all be able to agree on something as fundamental as the core OS. We should shoot for something much more realistic.

This is where hypermedia comes in. The most distributed system ever built — the World Wide Web — uses a hypermedia type (HTML) as its engine. There is a great opportunity to create a hypermedia format for car APIs that will energize the space just like HTML did for the Web. I believe this format could be based on an existing, generic type such as: Uber, HAL or Siren. This would be similar to the way the Collection.Document type was created for the news media industry, based on Collection.json.

2. Adopt a Standard API Security & Identity System
The prospect of connected cars getting hacked creates enormous anxiety. But connected car security can be addressed quite simply by adopting a security framework based around compartmentalization and standards-based access control.

In this context, “compartmentalization” means that core functions of the vehicle should be highly guarded. Specifically, no third-party app should have access to core driving functions like handling and braking. Meanwhile, a standards-based access control framework like OAuth will provide secure, granular access to specific system features. This would be similar to the way mobile apps currently ask for access to other parts of the device (GPs, contacts etc.)

3. Enable App Developers
Currently, only the lucky few are able to develop apps for connected cars. Generally, these are app vendors that have formal partnerships with car manufacturers. In most cases, developers can’t even get access to API documentation without a group of lawyers signing stacks of papers. The connected car space will not develop if it remains a tightly-held, closed system. On the contrary, manufacturers must build developer communities by providing the things that developers require: documentation; self-service portals; sandboxes; SDKs etc.

But That’s Not All
These are three immediate steps that can be taken to improve the connected car space significantly but as the space develops, we will have to focus not only on immediate requirements but also on the big picture. The connected car is a special case of the Internet of Things (IoT). The context of IoT is different enough that it requires a fundamentally different approach to system design and architecture. Hopefully, I will be able to delve into this context more in future.

Another aspect of the big picture is a good deal simpler: fun. If this space is going to develop as it should, manufacturers will have to make it fun for developers to experiment with the potential of automotive connectivity.

So, have fun out there!

June 9th, 2014

Jailbreak Your APIs

Jailbreak Your APIsAt this week’s Computers, Freedom & Privacy Conference, I gave a presentation titled Jailbreak Your APIs, in which I explored the concept of “linked APIs” and explained the potential these interfaces have for helping us create a freer, more open world. The global informational overload that we are constantly exposed to, can be overwhelming but ideas like linked APIs help us remember that the explosive surge of available data also brings us beautiful things such as transparency, openness and an unprecedented feeling of global connectedness.

We’ve never felt more connected to the rest of the world than we do now. Computers, mobile devices and the Internet have brought us closer than ever before. We now take it for granted that a person can be pretty much anywhere in the world and still get a real-time, front-row view of breaking news from half-way around the globe. While this disappearance of informational boundaries has surfaced many of our most polarizing differences, we still cherish our unprecedented ability to access information because access to information has always been our most powerful weapon for defending our rights and liberties.

In this context, the White House’s Open Data Policy (part of the Open Government Initiative) is particularly exciting. Never before has the American public had so much access to government information, at all levels. And all this happens directly through the Internet, in near real-time. The ability to access this information in a timely manner is crucial – we need access to information right when it is immediately relevant to guaranteeing our freedoms. This relates to my work with CA Layer 7’s API Academy because APIs supply the core technology for facilitating timely access to data.

Recent growth in the prominence of APIs is not simply a reaction to Open Government and Open Data. The API has organically become more important in recent years, due to our increasingly mobile lifestyles. APIs are vital to mobility because they connect our mobile devices to the cloud – specifically, to the datacenters that host the information and functionality that powers our apps. APIs have played an undeniably critical role in the mobile revolution of recent years. However, for APIs to play a similar role in the Open Data revolution, we need them to become much better.

The problem with APIs right now is that most of them are, at best, creating narrow windows into solid walls surrounding siloed data. Even the biggest, most well-known APIs (such as those provided by Twitter, Facebook and Google) to a large extent, only operate on the data that is within these organizations’ own databases. And most Government APIs don’t even allow any “write” functionality – they are strictly read-only.

In that sense, most current APIs create isolated, guarded data islands. This is very “anti-Web” — the World Wide Web was created in the spirit of decentralized equal participation. On the Web, everybody publishes everywhere, owns their data and then we have ways to reach that data through hyperlinks, through Google search and other methods. APIs have not really reached that stage of maturity yet. APIs are highly centralized, in terms of data storage and virtually none of them ever link to other APIs.

We need a new breed of interfaces: linked APIs, based on the same hypermedia design that we have on the rest of the Web. Such APIs will have the biggest impact for Open Data because they will link and make connections across datasets and organizational boundaries. Linked APIs are also very scalable, so they will be best suited to meeting the challenges of Big Data. After all, the Web is the largest, most distributed network of information humankind has ever created. We know the architecture of the Web can scale and linked APIs have the exact same architecture, with hypermedia as the engine.

For freedom of data, we really need more linked APIs. We can only truly have open and free data if we jailbreak the information out of the silos it is currently stashed-away in. Linked APIs provide us with keys to the data fortresses where large aggregators currently keep data. Linked APIs can ensure that our data isn’t stashed in centralized warehouses. Linked APIs represent the engine of data freedom on the Web. Let’s get the engine cranking!

May 29th, 2014

Toward a Lean API Strategy

Books on Lean Business Strategy“Lean”, “API” and “IoT” are probably the most hyped terms in our industry right now. Normally, I tend to blog about the latter two but – for a change – I would like to balance that out by talking about the former: the concept of Lean and how it relates APIs.

You have probably heard about or even read The Lean Startup by Eric Ries. And you may have noticed that this book sparked a whole cottage industry of Lean publications, like Lean Analytics, Lean UX and the widely-misunderstood concept of “minimal viable product”.

But few of you may have ventured back to explore the texts that laid the foundation for the Lean startup. The Four Steps to the Epiphany by Steve Blank, for example. Blank outlines a business process called “customer development”, which helps startups find “problem solution” and “product market” fit. Even fewer will have ventured right back to the very origins of the Lean concept: the Toyota Lean Production System with its emphasis on pull over push and ever-decreasing batch size towards one-piece-flow manufacturing. And we have not even touched upon the “theory of disruptive innovation” that Clayton Christensen outlines in The Innovator’s Dilemma or Rita McGrath’s concept of “discovery-driven planning” outlined in Discovery Driven Growth.

But the purpose of this post is not to provide a comprehensive reading list for those of you hoping to learn more about Lean and discovery-driven business strategies. My real goal is to explore if and how these concepts can be applied to API design best practices. However, if you are curious and want to know more about the books mentioned, I suggest you head over to a blog post I wrote for launchd.io. And for your next long-haul flight, you might want to consider starting with The Goal, which will provide you with some truly novel-like business reading.

Before I explore how Lean and API design come together, let me first make a confession – I got an MBA a couple of years back. I know that this is not going to win me any brownie points and I still prefer code to spreadsheets – no contest! But it goes some way to explaining why I think business and API are joined at the hip. The business value of an API does not come from the interface’s intrinsic technical features but from its ability to provide access to a business asset or service. APIs provide a technical means to do (more) business.

From this follows the assumption that API design and implementation need to focus on the intended business outcome. Which means that you must have a clear view of your business goals before you can start to implement your strategy. Unfortunately – in my experience – most of us on the technical side are not equipped to talk to the business side of the house (and they are seldom well prepared to talk with us). This is why I went back and got an MBA – so that I could learn to speak with “them” and build better products.

Starting with Toyota’s lean product development process, I began to see approaches and tools that could help bridge the semantic gap between the technical and business sides. I plan to share some of these with you in subsequent posts. I will start by discussing Alexander Osterwalder’s Business Model Canvas. To get a little background on what I’ll be talking about in my next post, I suggest that you read this post on ProgrammableWeb, where Mark Boyd uses the Business Model Canvas to analyze Walgreens’ QuickPrints API. Also, you might want to take a look at my Lean API Strategy presentation from the recent APIdays in Berlin.

May 23rd, 2014

Mapping the API Landscape

Mapping the API LandscapeThis week, I had the opportunity to deliver a “lightning talk” at the APIStrat Tech Un-Workshop at Gluecon 2014. The event was focused on two key topics: IoT and service description/discovery. I was in the service track and delivered a talk called Mapping the API Landscape. I won’t cover the entire talk here (BTW, the text has lots of links to information I could not discuss on stage this week) but did want to hit some key points.

What Google’s Self-Driving Car Tells Us
The Google car has been in the news again and a key point that was discussed at some length in these articles was the fact that the car depends on a very detailed map of the roadways. Right now, the car can only drive in the Mountain View, CA area since that is the only landscape mapped well enough for the car to navigate.

So, the Google car does not “discover” anything while driving. It actually recognizes intersections, traffic lights, etc. through a special representation of the landscape that contains all the right annotations. This reliance on a known map allows the car to navigate successfully between two points within that landscape – which is no simple feat, of course. Reacting to surroundings “at speed” takes serious computing power and that’s one of the reasons the Google implementation is so amazing.

Norman’s Action Lifecycle
The process of navigating from A to B is a goal-driven process that we see very often in nature. Ants, micro-organisms etc. all do this. Human-computer interaction expert Donald Norman calls this process the “action lifecycle” or “seven stages of action”.

This is how we learned to write GUI interfaces, too. Wait for a keystroke or button-click, process that action, affect the UI, then allow the user to evaluate the changes and decide if another action is needed. We also build Web servers this way. Wait for a request, process it, modify the backend (if needed) and reflect results back to the requestor. Game programming works like this, as well. But it’s rare to see the Web and mobile client applications that leverage enterprise APIs written in this way. They continue to look like single-minded bots that just go from A to B and ignore landmarks, incapable of actually reacting to surroundings.

Client Apps & Web Maps
Why are most client apps one-off implementations that are likely to break over time? It’s because client developers don’t have decent “maps” of the online landscape. And good maps are not just “photos” of the surroundings but heavily-annotated representations with recognizable symbols and landmarks. Most servers today just belch out JSON or XML with almost no recognizable symbols or signage (e.g. hypermedia controls).

So, what we need to do is create maps for devs to use so that they can build their own client applications against our APIs and solve their own problems. Client apps should be free to follow whatever route they wish within the maps – they shouldn’t be limited to following the path that server developers decide upon.

Let’s Make Maps!
Things like service description formats and discovery protocols are all ways to start creating more maps for client devs to rely upon. Using hypermedia in responses provides the symbols and signs client apps can use at runtime (like the Google car) in order to navigate in real time.

There are several description formats (see my paper Hold Your Nose vs. Follow Your Nose for more on this). In the book RESTful Web APIs, Leonard Richardson and I list close to 20 options for expressing hypermedia in Web responses. And more have come online since the book was published last year.

We have all the tools we need. We just need to use those tools to make more maps!