First and foremost, an API can’t be considered private if it is open for widespread public use, right?  A simple way of keeping an API private is to host the interface on a public network without explicitly advertising or documenting its existence.  This can work well initially but may lead to problems in the future. If your service is valuable enough that others want to get their hands on it, even an undocumented, unsupported, private API can easily end up becoming a depended-upon API for application developers, resulting in an outcry when the API publisher has the audacity to modify or retire its own service.

A better approach is to provide access control at run-time and restrict usage of your API to a few known parties. There are a great number of methods for protecting access to internal resources but the best ones are those that achieve a balance between ease of implementation and resistance to infiltration. Security at all costs can greatly increase the complexity of an interface and – in turn – the time required to complete the projects that depend on it. Instead, we need to implement access control that is practical. Thankfully, security protocols like SSL, HTTP Basic authentication and OAuth 2 are great for providing the basic level of access control needed to make it difficult for outsiders to use a private API. Bear in mind that there is much more to API security than simply validating identity but this is the minimum level needed to ensure a degree of privacy.

Although a private API’s developers are generally known to the publisher, the best private APIs utilize API portal components to provide self-service registrations and integration to their private developer communities. This can greatly reduce the friction involved in getting API integration-based projects started and reduce the overall project costs for B2B and mobile-based initiatives. In fact, many of the lessons of simplified design, documentation and administration learned from the public API world can be directly applied to private API management. While the ultimate goal may be different (driving efficient API usage for private APIs rather than far-reaching adoption of open APIs), the ways of getting there are largely the same.

A unique characteristic of private APIs is the need to manage groups of developers. Unlike the public API space, private API publishers will often define out of band contract terms before offering up a quick self-service integration mechanism for that team. This type of group-based role definition is particularly common in integration projects that occur between organizations and can stretch the limits of API portal software that has been built primarily for open API use. Ideally, an API portal should at least be capable of managing developers within groups, communities or organizational affiliations as part of the self-service registration process. Even better, the portal could  provide capabilities for managing whole communities as separate domains within the same infrastructure.

Designing a private API certainly requires a different perspective but the good news is that much of the knowledge around public API design can be directly applied to interfaces you want to keep secret. Of course, building the management and security capabilities required to expose the API to your trusted parties can be daunting but that is why a great API management portal and gateway combination can save the day.

At the Layer 7 API Academy, we’ve had a few requests from API designers who are seeking strategies for handling large amounts of data in API responses.  Pagination is the most common method for addressing this scenario. Pagination, which is very common on the Web, allows API architects to conserve resources, improve response times and optimize the user experience. It’s a way of splitting up data into “pages” and is used in just about any API that returns collections of data.

I’ve released a short video tutorial titled Use Pagination in Web API Design to introduce the ins and outs of the interface. This video provides a crash course explaining pagination and outlining how to use it effectively in the design of Web APIs. I couldn’t fit all the implementation considerations I wanted in this six-minute tutorial, so watch out for a follow-up video on the subject.

That’s why we at the API Academy stress the importance of making rational decisions rather than irrationally selecting design patterns based on emotion or trends.  We want you to choose your design elements rather than picking them from the latest set of formats or technologies that you’ve heard about.

And that’s why we’re working on a series of tutorial videos, as my colleague Mike Amundsen recently announced. The first of these videos, titled The API Interaction Model – An Introduction, provides an overview of  a design process that will help you consider your user’s perspective in order to make effective design choices later. The ideas I discuss in this video are rooted in user-centered design processes that have been very effective in the software and product design worlds.

If you’re currently designing an API, invest five minutes and watch the video.  It should be time well spent.

It’s a great story but it’s really only half the story. Web API experts regularly acknowledge the existence of a “private” or “closed” API market. In fact, many of us believe that if the number of private APIs in use could be cataloged it would dwarf the 7,000 or so APIs that are published on the ProgrammableWeb site.

As with many of the terms in the API world, there isn’t a concrete definition of  “private API”. In general, a private API has these characteristics:

1. It provides a language-independent interface that is made available via Web protocols.
2. It’s access is limited to a specific set of known developers or organizations.
3. It is not marketed to the general public nor is its documentation made publicly available.

Further to this, we can divide private APIs into three major buckets:

1. Internal APIs that exist within the organization’s borders (for example, SOAP-based interfaces within an internal Service Oriented Architecture).
2. Business-to-business (B2B) APIs that enable organizations to integrate with external companies.
3. Backend APIs that drive mobile, Web and device-based applications.

With this definition we can see that there are a great many integrations that must already exist. Enterprises have been building SOAP and B2B-based connectivity for years and have accumulated healthy inventories of private APIs.

In addition, the headlong rush towards the world of mobile is driving the creation of new externally-facing APIs to help corporations reach their customers. As I’ve talked about before, many organizations wish to retain control over the development of these applications and they can do this with private APIs.

If IT teams have been building these types of in-house connectivity solutions for so many years, there shouldn’t be much room left for innovation or improvement, right?

Not quite. Unlike those who build private APIs, public API designers are motivated by the need for their interfaces to be chosen out of the mass of APIs that are available to their prospective users.  This difference in motivation has created a massive impact on how public APIs are designed and managed. Architects responsible for private APIs have a great opportunity to learn from the public API world by adopting design strategies devised to drive adoption, in a controlled manner.

A good reason to take a developer-centered approach to private API design is the development cost associated with building applications that utilize the interface.  A well-designed private API can reduce the project costs for application development as well as for maintenance and upkeep of the integration.  Good design isn’t easy but it pays off – even when the audience is limited.

Many enterprises are implementing a “private for now and public later” API strategy.  It is a great idea but that doesn’t mean architects shouldn’t strive to incorporate great API design and a solid management solution.

In my next post, I’ll dive into private APIs in more detail and talk about some of the specific challenges that arise when building closed interfaces and how these challenges can be addressed with management solutions.

We in the API design world have a great opportunity to learn from our brethren in the product design world. The user-centered design approach for products has paid great dividends for those who can understand and apply the idea to their interfaces. The goal is almost stupid in simplicity – design products that your users will enjoy. But, as always, the challenge is in translating a simple concept into real strategies, methodologies and practices that do not lose that fundamental goal while staying applicable to unique marketplaces.

In our world of API design, most of us understand that machine-to-machine integration still involves a human – the developer who develops the client code. That developer – the one who makes or breaks us by deciding to use an API – is our user. While product designers talk about improving user experience, we talk about improving the developer experience.

But how does this actually happen? What do we specifically need to do in order to create APIs that are enjoyable to use? Indeed, what does enjoyable even mean in this context? This developer/API publisher relationship is a unique one and the product-based, user-centered design and human/computer interaction models cannot just be airlifted in. They need to be massaged and transformed so they are applicable to the Web API world, without losing the potential value inherent in a user-focused design.

I hope to explore these ideas over the coming months and come up with recommendations for how we can build API solutions that deliver on the promise of improved developer experience (or DX). I’ll dive deeper into the world of user-centered design and discuss methods for translating these concepts from the world of product design into our API design domain.

I was particularly impressed by the range of industries and organizations that were represented in the audience. We had developers from large enterprise shops, specialized Internet-focused start-ups and even a few entrepreneurs just getting started. I think this range of interest is indicative of the value of Web APIs for all and bodes well for a continued investment in designing great APIs, rather than just chucking them out into the ether.

Next up on the tour is our Paris API Workshop taking place tomorrow (Tuesday, October 16).  As always, we have a great set of speakers lined up, with Martin Duval from bluenove talking about building developer outreach programs and Benoit Herard from Orange Labs discussing their API launch. France has a  great start-up culture and a reputation for enterprises like Orange driving innovation, so I’m expecting good conversation, some excellent API Management presentations and – if I’m lucky – some great wines.

Over the last few years, we’ve seen explosive growth in open APIs across North America, lead of course by the avant garde Internet companies on the West Coast. The API Management industry has focused much of its attention on the US market but the Web API movement has definitely made its way to other markets and the push towards mobile and device-based applications is clearly having an influence on enterprise architectures.

Western Europe has had a strong influence on the API scene, with notable government and enterprise organizations diving wholeheartedly into the collaborative, developer-focused open API space. London, in particular, has developed a thriving technology scene with tons of hackathons, codeathons, meetups and start-up companies trying to change the world or at least get rich trying.

At the moment, the open API scene in India is still in its infancy and I’m looking forward to helping the concept blossom in whatever way that I can. As you may be aware, the number of mobile devices being used in India is mind-boggling and the ratio of mobile-use-to-desktop-computing is much higher than in North America or Western Europe.  This quantity of mobile client platforms, combined with the large number of motivated developers on the scene, makes this a very intriguing open API marketplace. I can’t disclose any details on the nature of the project yet… but I’m hoping to to have exciting news to share in the near future, so stay tuned.

I’ve spent most of the summer in North America, for a variety of reasons and I’m excited that I will finally be getting back home to the UK so I can re-engage with the European API and mobile scene. We have some great Layer 7 API workshops scheduled across Europe over the next few months and hopefully we will uncover a few new and noteworthy European API publishers while we are on tour.

It took time and some adaptation for concepts like open source software, social media integration and viral marketing to become part of the enterprise world and I believe that opening up Web APIs will require a similar shift in mindset to work on the enterprise stage. The biggest ships take the longest to turn but modern businesses (even the most risk-averse) must be open to leveraging new technologies and architectural philosophies in order to avoid being left behind.

The buzz around Web APIs has definitely piqued the interest of big business and large enterprises have dipped their toes into its waters with the release of a few compelling APIs over the last year.  But, along with the excitement generated from opening new consumer channels and new avenues for innovation, there is still a  prevailing sense of danger associated with the API movement.

For many enterprises,  there is a fear that publishing APIs means giving up control of their services and data to an army of anonymous 16 year-old mobile developers. After all, who wants their carefully crafted brands and products to end up at the mercy of the masses? We’ve seen marketing experiments with “crowd sourcing” produce some interesting results in the past, so there is reason to be cautious when opening up the doors for collaboration in any form.

Of course, the good news is that the challenge of controlling APIs can be elegantly addressed with a strong API Management system. At Layer 7, our SecureSpan API Proxy gives enterprise customers the tools they need to maintain control over how content and services are used, allowing publishers to lock down APIs as much as they want.

However, publishers will also need to ensure that they provide enough accessibility to their API libraries or they will run the risk of exposing wonderful APIs that sit unused, waiting for developers to utilize them. APIs are only useful when they are used and a closed-door policy will not encourage anyone to sign up. That’s why we also offer the Layer API Portal, which is designed to facilitate developer community outreach and secure developer onboarding.

Making APIs attractive to the developer community is the key to increasing usage and it is becoming clear that developers want stability and control in the APIs they use. For example, Twitter’s continued restrictions on API usage and Facebook’s closure of the face.com face recognition API have created a small wave of backlash amongst their developer communities. While it’s not enough of a storm to make much of a dent in the uptake of Twitter or Facebook APIs,  application developers are realizing that building their apps based on APIs from which they may lose access is ultimately a losing proposition.

This is good news for larger enterprises as it signals a growing level of maturity in the API market and the need for stable, fairly-priced APIs that can support apps in the longer term. A set of well-designed, secure APIs with a well thought out revenue model is exactly the right fit for the large enterprise world.

So, are open APIs too open for enterprises? Probably. But enterprises will need to adapt or risk being unable to reach their customers as the device revolution continues at its explosive pace. Conversely, launching a poorly-designed API library just to get it out there can be an equally devastating misstep. Organizations need to think carefully and plan their API strategies in order to find the perfect balance between control and accessibility.

It isn’t easy for enterprises to embrace open APIs but when the risks are managed properly with a well-built API Gateway, developer portal and API strategy, the rewards can be immense.

Answer: Neither —  they are really only useful as a whole. A client without a server is usually little more than an non-functional wire frame and a server without a client is simply unrealized potential. Bring them together though and you have something of lasting value. So, neither matters more and each actually matters a lot less than half.

In the API world, this is an easy point to miss. The server side always wields disproportionate power by virtue of controlling the API to its services and this can easily foster an arrogance about the server’s place in the world. This effect is nicely illustrated by Twitter’s recent missteps around developer management.

The problems for Twitter all began with a blog entry. Blogs are the mouthpiece of the platform. Tucked away within an interesting entry about Twitter Cards and the potential to run applications within tweets (something that is genuinely exciting), can be found a restatement of an early warning to developers:

“(D)evelopers should not ‘build client apps that mimic or reproduce the mainstream Twitter consumer client experience.’”

Ominous stuff indeed. This was quickly picked up on by Nick Bilton writing in the New York Times Bits blog, who pointed out that the real problem is that Twitter just isn’t very good at writing client-side apps that leverage its own API. Stifling competition by leveraging the API power card can only alienate developers — and by extension the public, who are left with a single vendor solution. Suddenly, it feels like the 1980s all over again.

This ignited a firestorm of concern that was well summarized by Adam Green on ProgrammableWeb. Green acknowledged that API change is inevitable but pointed out that this is something that can be managed effectively — which is not what Twitter is doing right now.

The irony of the whole thing is that, in the past, by exercising its power position, Twitter has actually made great contributions to the API community. In mid 2010, Twitter cut off basic authentication to APIs in favor of OAuth, a drop-dead event that became known as the OAuthcalypse. Hyperbole aside, in terms of actual impact on the populace, this cut over made even Y2K look like the end of days. Given a tractable challenge, developers cope, which is really Green’s point.

What is important to realize is that API Management isn’t technical but social. Win the community over and they will move mountains. Piss them off and they will leave in droves for the next paying gig.

The thing I always remind people is that as a trend, APIs are not about technology; they are a strategy. Truth is, the technology is pretty easy — and that’s the real secret to API’s success. You see, the communications are never the thing; the app is the thing (and that is what WS-* missed). Maintaining simplicity and a low barrier to entry counts for everything because it means you can get on with building real apps.

Now, I can give you the very best infrastructure and tools to facilitate API community. But how you manage this community… Well, that is where the real work begins and — in the end — it’s all a lot less deterministic than we technologists like to admit. People are hard to manage but communities are even harder.

If there is a lesson here, it is that APIs are really about potential and that potential can only be realized when you have two sides — client and server — fully engaged. Mess this one up and you’re left with just a bunch of unused interfaces.

One thing that is encouraging and has the potential to trigger disruptive change to the delivery of government services in the US is the recent publication Digital Government: Building a 21st-Century Platform to Better Serve the American People. The word to note here is platform –  it seems that government has taken a page from Facebook, Twitter and the others and embraced the idea that efficient information delivery is not about a carefully-rendered Web page but instead is really a logical consequence of developing an open platform.

I confess to some dread on my first encounter with this report. These publications are usually disheartening products of weaselly management consultant speak refined through the cloudy lens of a professional bureaucrat (“we will be more agile”). But in this instance, the reverse was true: this report is accessible and surprisingly insightful. The authors understand that Mobile+Cloud+Web API+decentralized identity is an equation of highly interrelated parts that, in summation, is the catalyst for the new Internet renaissance. The work is not without its platitudes but even these it bolsters with a pragmatic road map identifying actions, parties responsible and (gasp) even deadlines. It’s actually better than most business plans I’ve read.

Consider this paragraph clarifying just what the report means when it calls for an information-centric approach to architecture:

An information-centric approach decouples information from its presentation. It means beginning with the data or content, describing that information clearly, and then exposing it to other computers in a machine-readable format—commonly known as providing web APIs. In describing the information, we need to ensure it has sound taxonomy (making it searchable) and adequate metadata (making it authoritative). Once the structure of the information is sound, various mechanisms can be built to present it to customers (e g websites, mobile applications, and internal tools) or raw data can be released directly to developers and entrepreneurs outside the organization. This approach to opening data and content means organizations can consume the same web APIs to conduct their day-to-day business and operations as they do to provide services to their customers.

See what I mean? It’s well done.

The overall goal is to outline an information delivery strategy that is fundamentally device agnostic. Its authors fully recognize the growing importance of mobility and concede that mobility means much more than the mobile platforms — iOS and Android, among others — that have commandeered the word today. Tomorrow’s mobility will describe a significant shift in the interaction pattern between producers and consumers of information. Mobility is not a technological instance in time (and in particular, today).

But what really distinguishes this report from being just a well-researched paper echoing the zeitgeist of computing’s cool kids is how prescriptive it is in declaring how government will achieve these goals. The demand that agencies adopt Web APIs is a move that echos Jeff Bezos’ directives a decade ago within eBay (as relayed in Steve Yegge’s now infamous rant):

1. All teams will henceforth expose their data and functionality through service interfaces.

It was visionary advice then and it is even more valid now. It recognizes that the commercial successes attributed to the Web API approach suggest that just maybe we have finally hit upon a truth in how system integration should occur.

Of course, memos are easy to ignore — unless they demand concrete actions within limited time frames. Here, the time frames are aggressive (and that’s a good thing). Within six months, the Office of Management & Budget must “Issue government-wide open data, content, and web API policy and identify standards and best practices for improved interoperability.” Within 12 months, each government agency must “Ensure all new IT systems follow the open data, content, and web API policy and operationalize agency gov/developer pages” and also “optimize at least two existing priority customer-facing services for mobile use and publish a plan for improving additional existing services.”

If the recent allegations regarding the origins of the Stuxnet worm are accurate, then the President clearly understands the strategic potential of the modern Internet. I would say this report is a sign his administration also clearly understands the transformational potential of APIs and mobility, when applied to government.