May 23rd, 2014

Mapping the API Landscape

Mapping the API LandscapeThis week, I had the opportunity to deliver a “lightning talk” at the APIStrat Tech Un-Workshop at Gluecon 2014. The event was focused on two key topics: IoT and service description/discovery. I was in the service track and delivered a talk called Mapping the API Landscape. I won’t cover the entire talk here (BTW, the text has lots of links to information I could not discuss on stage this week) but did want to hit some key points.

What Google’s Self-Driving Car Tells Us
The Google car has been in the news again and a key point that was discussed at some length in these articles was the fact that the car depends on a very detailed map of the roadways. Right now, the car can only drive in the Mountain View, CA area since that is the only landscape mapped well enough for the car to navigate.

So, the Google car does not “discover” anything while driving. It actually recognizes intersections, traffic lights, etc. through a special representation of the landscape that contains all the right annotations. This reliance on a known map allows the car to navigate successfully between two points within that landscape – which is no simple feat, of course. Reacting to surroundings “at speed” takes serious computing power and that’s one of the reasons the Google implementation is so amazing.

Norman’s Action Lifecycle
The process of navigating from A to B is a goal-driven process that we see very often in nature. Ants, micro-organisms etc. all do this. Human-computer interaction expert Donald Norman calls this process the “action lifecycle” or “seven stages of action”.

This is how we learned to write GUI interfaces, too. Wait for a keystroke or button-click, process that action, affect the UI, then allow the user to evaluate the changes and decide if another action is needed. We also build Web servers this way. Wait for a request, process it, modify the backend (if needed) and reflect results back to the requestor. Game programming works like this, as well. But it’s rare to see the Web and mobile client applications that leverage enterprise APIs written in this way. They continue to look like single-minded bots that just go from A to B and ignore landmarks, incapable of actually reacting to surroundings.

Client Apps & Web Maps
Why are most client apps one-off implementations that are likely to break over time? It’s because client developers don’t have decent “maps” of the online landscape. And good maps are not just “photos” of the surroundings but heavily-annotated representations with recognizable symbols and landmarks. Most servers today just belch out JSON or XML with almost no recognizable symbols or signage (e.g. hypermedia controls).

So, what we need to do is create maps for devs to use so that they can build their own client applications against our APIs and solve their own problems. Client apps should be free to follow whatever route they wish within the maps – they shouldn’t be limited to following the path that server developers decide upon.

Let’s Make Maps!
Things like service description formats and discovery protocols are all ways to start creating more maps for client devs to rely upon. Using hypermedia in responses provides the symbols and signs client apps can use at runtime (like the Google car) in order to navigate in real time.

There are several description formats (see my paper Hold Your Nose vs. Follow Your Nose for more on this). In the book RESTful Web APIs, Leonard Richardson and I list close to 20 options for expressing hypermedia in Web responses. And more have come online since the book was published last year.

We have all the tools we need. We just need to use those tools to make more maps!

April 24th, 2014

SDKs Work Until They Don’t

SDKs and APIsOver the last couple of months, I’ve had many great opportunities to road test my findings on SDKs vs. APIs, with a wide variety of audiences – at our API Academy Summits in New York and London, on tour with the Nordic APIs team in Stockholm and Copenhagen and at my most recent API Workshop in Istanbul. Given the apparent relevance of the topic and the lively discussions I’ve had around it, I’d like to take this opportunity to summarize some of the insights and recommendations that have come up.

If you’ve been following our blog, you will remember that I started this topic with the observation that leveraging an API increasingly seems to involve using an SDK rather than the API itself. I followed up with a post talking about my decidedly mixed experiences of trying to use SDKs. My own experiences inspired the headline of this post: SDKs work until they don’t.

So, what are the main motivations to invest in an SDK, from an API-provider perspective?

  • Simplifying API design by extracting business logic into the client
  • Maximizing scalability by exploiting client-side processing
  • Empowering developers to leverage the API more quickly
  • Presenting an optimized client from a target-platform perspective (e.g. for mobile connectivity or constrained hardware)
  • Providing a strongly-typed presentation of the API in a variety of programming languages

Let’s contrast all this with the main drawbacks of SDKs:

  • Picking which platforms, languages and frameworks to support – some of your target developers are going to end up disappointed
  • Relying on third-party frameworks – any developer who has to integrate with two or more APIs leveraging the same framework at different version levels is bound to experience some headaches, for example
  • Adding carry-on weight of unused functionality to the application
  • Incurring long-term support costs for the SDK

But to me, the biggest risk of a SDK-first approach lies in making API design an afterthought. We have come to this point in the API evolution because pragmatic REST introduces just enough constraints to force us to think about how we can abstract the underlying business asset into a resource-based model restricted to CRUD-style interactions. An SDK-first approach might tempt us to go back to a RPC-style API design mirroring the backend implementation, resulting in all the inherent integration complexities we had with Web services.

So, if and when you decide on a SDK, keep the above in mind. You might still come to the conclusion that you need an SDK in order to quickly onboard developers or provide the best client for your API but at least you will be able to consciously weigh the benefits against the drawbacks.

If you want to dig more deeply into the subject, I can highly recommend the following podcasts, articles and blog posts, which helped greatly in forming my own opinions:

April 17th, 2014

Next API Tech Talk: Linked APIs

Linked APIsThe challenges faced by today’s software architects go far beyond the familiar. “Big Data” means more than managing petabytes of data – it requires dealing with data-sets that span organizational boundaries. Likewise, the term “distributed system” no longer refers to just a multi-tier architecture or cloud deployment – it usually involves the connection of non-heterogeneous systems across multiple organizations.

On Thursday April 24, I’ll discuss these challenges as part of Layer 7’s latest API Tech Talk. I’ll be using this opportunity to explore how architects can leverage “linked APIs” to handle Big Data sets and distributed systems that cross organizational, technological and cultural boundaries, breaking through data silos in order to better integrate information. Interested? Just add the Tech Talk to your calendar and go to api.co/L7live at 9am PDT (12pm EDT) next Thursday.

I’ll also be taking your questions on linked APIs, Big Data, distributed systems, open source and anything related, so please don’t hesitate to join in. You can submit your questions now by email or you can chat with me or tweet them at me on the day. This will be my first Tech Talk since joining the Layer 7 API Academy and I’m really, really looking forward to a lively discussion. See you on April 24!

April 4th, 2014

API Academy Goes to Asia

API Academy in AsiaStarting April 4, I’ll be on the road for close to two weeks. Along the way, I will have the honor of bringing the API Academy message of developer-focused, enterprise-scale API design and implementation to the cites of Seoul, Tokyo and Singapore. In each of these cities, we’ll be hosting a free half-day seminar covering some of the most popular topics the API Academy’s private, onsite training offers to companies the world over.

I will also have the chance to do some additional presentations and make new connections while on this trip. As much as I enjoy the workshops, it is the chance to connect with people I’ve only known online and to meet new ones that really makes these trips a great experience.

WWW 2014 in Seoul
While in Seoul, I will have the honor of presenting a peer-reviewed paper to the WS-REST2014 workshop, which is part of the World Wide Web Conference in Seoul. It is not often that I get the opportunity to speak at events of this caliber and I am also looking forward to catching up with several people who work on W3C projects – people I rarely get to meet in person.

There will also be an informal meet-up in Seoul on the evening of April 8 near the COEX complex where the WWW 2014 event is to be held and not far from the API Academy public workshop on the April 9. I don’t have all the details yet and promise to post them as soon as I have them.

RESTful Web APIs in Tokyo
I am very excited to announce that I will be attending a RESTful Meetup in Tokyo the evening of April 12. This was organized, in part, by a group of people who have also been hosting a bi-weekly reading group for the book RESTful Web APIs.

This group popped up last year to allow people to come together and translate the English-language edition of RESTful Web APIs in “real time” by taking turns reading the content and then discussing it as a group. Leonard Richardson and I are very grateful for this kind of enthusiasm and I am looking forward to meeting some of the people behind this cool project.

Singapore
I will arrive in Singapore on Monday, April 14 and don’t have any additional meetups scheduled yet. If you’re in Singapore and want to set up something, ping me and let’s see if we can get something going while I am in town for the public workshop on April 15.

Okay, Let’s Go!
The chance to visit customers, developers and designers in Seoul, Tokyo and Singapore really has me energized. If you’ve not yet signed up for one of the public workshops, please do. And come up and tell me “hello”. I’d love to hear about what you’re working on and how the API Academy can learn from your experience and help you reach your goals for building great applications for the Web and the enterprise.

(This post was originally published on my personal blog.)

February 27th, 2014

New API Academy Team Member: Irakli Nadareishvili

Irakli NadareishviliThe API Academy team has a new member: Irakli Nadareishvili who has joined CA Layer 7 as Director of API Strategy. Before joining CA, Irakli served as Director of Engineering for Digital Media at NPR, which is noted for its leadership in API-oriented platform design. He has also participated in the creation of the Public Media Platform, worked with whitehouse.gov and helped a number of major media companies develop publishing solutions using open source software.

I recently sat down with Irakli to discuss what he has in mind as he joins API Academy.

MM: You once told me that you believe the future of Big Data is “linked APIs”? That sounds intriguing. Tell me more about it.

IN: In most people’s minds, “Big Data” is synonymous to “very large data”. You may hear: “Google-large” or “Twitter-large” or “petabytes”. The Wikipedia definition of Big Data is slightly more elaborate:

“Big data is the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications”.

In my work, I see the “complex” part of the definition becoming more important than the size. We have gotten pretty good at taming the large sizes of data. Tooling for horizontal partitioning and parallel processing of large data sets is now abundant. Still, most Big Data sets are contained and processed in the isolation of single organizations. This is bound to change very soon. The end of siloed Big Data is near: I believe that the next phase of Big Data challenges will have to do with data sets that cross organizational boundaries.

APIs will play a major role in this. Web APIs represent the most effective available technology that allows data to cross organizational boundaries. APIs efficiently connect and link data at a distance.

MM: Can you give an example of what you mean by “data sets that cross organizational boundaries”? And what challenges do these pose?

IN: You see, a lot of people have the notion that the data they need to process can be stored in a database maintained by a single organization. This notion is increasingly inaccurate. More and more, organizations are having to deal with highly-distributed data sets.

This can be very challenging. The infamous healthcare.gov is a good example of such a distributed system. The main technical challenge of implementing healthcare.gov’s backend was that it had to integrate with data in many existing systems.

The $500 million initial public fiasco of healthcare.gov is also a vivid indication of just how complex it is to build truly distributed systems. Practically the only successful implementation of such a large, distributed information system is the World Wide Web. There’s a lot we can learn from the architecture of the Web. It’s a battle-tested blueprint for building distributed systems at scale.

I believe the Big Data challenges of the future will be solved at the intersection of APIs with Web/hypermedia architecture, linked data and what we currently call Big Data tooling. I call this intersection “Linked APIs”, to differentiate it from the current, siloed state of most Web APIs.

MM: What practical advice would you give to the developers of future Big Data APIs?

IN: I think the most important thing is that we need to stop falsely assuming all of the API data is local data. It is not. Despite the name, an API for a distributed system is really not a “programming interface” to local data/assets. Rather, it is a programmable data index. Think of APIs as a programmable search index for a distributed collection of data sets.

I don’t like to think of the term “API” as an abbreviation anymore. Maybe it was one a while ago but it has since evolved way past that. Much like IBM doesn’t think of itself as “International Business Machines” anymore, APIs aren’t merely “application programming interfaces”. Most of what IBM does these days isn’t even necessarily about “machines”. Likewise, most of what we need out of APIs isn’t about any single application or an interface to it.

MM: Big Data represents one important challenge for computing today. What about IoT?

NN: The Internet of Things is already here, in small ways. The IoT we have today consists of a vast number of Web-connected devices, acting as sensors, sending myriads of signals to the cloud. That, by the way, is what creates many Big Data challenges. The future is much more interesting, however. Once the connected devices start engaging in peer-to-peer interactions, bypassing central authority, we will enter a significantly different realm. The most important challenge in that world, from my perspective, will be identity. Identity is always key in distributed systems but especially so in peer-to-peer networks.

MM: What excites you the most about your new role at Layer 7?

IN: Thank you for asking this question. I will start by telling you what terrifies me the most. The API Academy and Layer 7 teams represent a gathering of  ”scary” amounts of world-class brainpower and expertise in the API space. It is extremely humbling to be part of such a distinguished group.

Obviously, it also means that there is a lot of very fundamental thinking and innovation that happens here. Especially now that Layer 7 is part of CA Technologies, there’s really very little that we couldn’t accomplish if we put our hearts to it. That feels extremely empowering. I really care about all things related to APIs and distributed systems, the role they can play for the future of technology. I am super excited about the possibilities that lie ahead of us.