Consider the case of a small business offering a cloud-based service product for a niche market.  If this business chose to invest in a well-designed, developer-centric API, at a minimum they could expect:

• A reduced learning curve for developers consuming the interface
• A reduction in troubleshooting time
• Increased interest from their developer community

For most audiences, these are goals worth achieving.  Indeed, this is why we emphasize good design for APIs in the first place – the benefits fit remarkably well with the main reasons for embarking on an API strategy: reduced cost and increased adoption.

But, in my conversations with connectivity professionals from larger organizations, it is apparent that not all service vendors see the value in investing in this type of design effort.  Developers and architects are bursting with  tales of forced integration with service providers who have simply thrown an ugly or barely functioning interface on top of  core component.  It is in these scenarios that we hear about laughable attempts at implement whichever API styles and features are in fashion.  ’REST’ and ‘security’ become sales-worthy buzzwords that don’t live up to their promise when developers get their hands on the actual interface when real project work commences.

In the majority of these cases, technical teams have very little say during the procurement process of outsourced and cloud-based services.  In effect, these API providers don’t need to design for their developer audience because they aren’t critical to succeeding.  For many years a sound strategy for selling cloud-based products has been to sidestep technical teams and engage directly with the business.  It’s frustrating that technology teams are often still left with the responsibility for reducing integration costs regardless of the lack of sophistication in the APIs that they are tasked with connecting to.

Thankfully, the wealth of knowledge and connectivity products in the enterprise space allows these teams to reduce the impact of bad design on the overall project and organization.  Components such as API proxies can be used not only to build a facade for APIs that are being exposed, but also to provide abstraction for services that are being consumed.  In essence, the design burden can shift from the service provider, to the enterprise developer who wraps a poorly designed interface in a more consumable, developer-friendly API for the rest of the organization to use.

As a whole, the scenario makes sense.  Well designed products are based in part a designer’s empathy for their users.  Good design involves perceiving a product from a user’s view point along with an understanding of the impact that design decisions will have on the user base.  However, an organization that builds an API as an afterthought for developers, who are only viewed as a means to an end, will likely produce a poor API.

Ultimately, building a technology business based on developer apathy is a bad idea.  The industry shift towards API product based integration is empowering technology teams at all levels and services that continue to ignore the needs of their developers will eventually be ousted from the market.  In short, good design is only a waste of time when you don’t care about your users.  If  in fact you do care about the developers who will be using your API product, then you need to invest in designing your API with their point of view in mind.

The challenge is that it’s really difficult to find people who are as great at documenting systems as they are at designing them. As a convenient shortcut, many API designers use tooling to auto-generate documentation. This often means exporting machine-readable interface description files like WSDLs and WADLs based on some type of configuration data entered into a development or testing tool. Assets like these are great for driving programmatic components on both the client and server side but they have limited value otherwise.

WSDL files, in particular, are popular in the SOA space because they allow client developers to auto-build proxy classes that can be invoked in the RPC style that is prevalent for SOAP-based integration. This advantage is diminished in the HTTP API world as we have moved away from this document binding style of interface to less structured forms of integration. But even putting this distinction aside, WSDLs have never provided an effective means of documenting SOAP systems and WADLs are no better.

Effective documentation implies effective communication. In the vast majority of situations, a standards-based XML description of your interface is not going to cut it. Application developers need to understand much more than the names and parameters of your API if they hope to build real applications in reasonable time-frames. This means you need to create documentation fit for developers – in other words, documentation built for humans.

Your documentation will act as a navigation system through the complexities of your API. Simply providing a WADL is the equivalent of providing a set of GPS coordinates to a tourist in your city. With the right tools, they may get there eventually but a human-readable map would provide a much richer and simpler experience. If you care about the developer experience for your API, you’ll spend some time and effort writing documentation that works.

How does Layer 7 help me manage a range of devices across IoT?
IoT is an opportunity for CA and Layer 7 to bring together identity, access and API Management.  To paraphrase a comment on a recent Gigaom article: Everything with an identity will have an API and everything with an API will have an identity.

With so many “things” potentially accessing APIs, what are some strategies for securing these APIs across such a breadth of consumers?
Identify, authenticate and authorize using standards. API for IoT means managing identity for many devices at Internet scale.

How will API discoverability work with the vast number of things, especially if we see REST as the primary communication style?
I reached out to my colleague Ronnie Mitra for this answer. Ronnie pointed out that, in the past, standards like UDDI and WSRR promised to provide service registries but that didn’t really work out. Nowadays, we see lots of independent human-oriented API registries and marketplaces that might have more chance of surviving. There are even some runtime discovery solutions like Google’s discovery interface for APIs and the use of HTTP OPTION to learn about APIs. At the moment, lots of people are trying lots of things, unsure of where it will all end up. It would be interesting to dive deeper into why we need discoverability to power IoT and when that discoverability has to take place.

How can API security get easier when API demand grows exponentially? There’s a big disconnect.
It doesn’t get easier. Transport-level security is reasonably well understood but endpoint identity and trust will be challenging.

Where will the intelligence be in IoT? Will there be some form of on-site intelligence, so that core functionality continues even if the connection is lost? Or will all intelligence be cloud-based?
It depends on whether you design for centralized “hub and spoke” or decentralized “domains of concern”. The former is responsible for correlating data and events within the domain whereas the latter is responsible for communicating with other domains (I owe this concept to Michael Holdmann’s blog). “Domains of concern” design communicates with different domains for different purposes –  in an apartment for home automation, in an apartment building for HVAC, in a city block for energy generation/consumption, in a city for utility grid etc. Emergencies or out-of-bound signals are handled like exceptions and are bubbling up through the domains until intercepted. But most things will serve an inherent purpose and that purpose will not be affected by the absence of any connectivity. There will be intelligence within the core of each domain as well as at the edges/intersections with other domains.

What is the best way to overcome fear of exposing data via APIs in an enterprise?
You need to identify a business opportunity. Unless you know what business impact you are trying to archive and how you will measure it, you should not do it.

Does IoT require a strong network or big data or both?
Not a strong network but ubiquitous connectivity. Not big data but sharing/correlating data horizontally between distinct vertical silos.

What significance (benefits/drawbacks) do the various REST levels have with respect to the Internet of Things (connecting, monetizing etc.)?
I had never heard of levels of REST and had to look it up. Turns out the levels are: resources, verbs and hypermedia. Hypermedia would allow you to embed long-lived clients, which could adapt to changes in API design. But it is actually the data or service behind the API which is monetizable, not the API itself. The API is just the means to an end.

How will IoT evolve? And more importantly how can enterprises solve the security and privacy issues that will arise as IoT evolves?
Culturally, the European regulators will try to put privacy regulations in place sooner rather than later whereas the North Amercian market will initially remain largely unregulated until some abuse prompts the regulator to step in. In Germany, the federal regulator tries to stay ahead of the market and recently published a security profile for smart meters. Personally I would look at designing M2M and IoT applications assuming that endpoint data is inherently unreliable and that I can not necessarily trust the source. But that is very broad guidance and may or may not be applicable to a specific use case.

As we create API frameworks that interact with sensors and control objects in the IoT what/who are the best organizations to follow to learn about new protocols that we should be preparing to handle, such as CoAP etc?
Here are some suggestions:

• IPSO Alliance
• ETSI M2M
• ITU M2M
• oneM2M
• Eclipse M2M

How close are we to having a unified platform for IoT application developers and who is likely to be the winner among the competing platforms?
Chances are there won’t be a winner at all. You have companies like Axeda, Exosite, Gemalto, Digi, Paraimpu, BugLabs, ThingWorx, SensiNode, deviceWISE and more. You have industry working groups like Eclipse M2M and various research efforts like SPITFIRE project, Fraunhofer FOKUS, DFuse and many others. The Eclipse M2M framework is probably a good choice to start with.

Even assuming ubiquitous and common networking (e.g. IPv6 on the public Internet) – how will the IoT identify peers, hierarchy and relationships?  
I think there is a huge opportunity for identity companies like CA to figure this out. Take a look at EVRYTHNG as one of the few startups in that space. Meanwhile, the folks over at Paraimpu are trying to tackle this challenge by combining aspects of a social network with IoT.

“Say you have an end point for validating address; call it /validateAddress. Now this endpoint is called from two work flows.

1. When a user updates his account settings (changes a new address)
2. When a user tries to buy a product and enters the shipment address

In both cases the /validateAddress should give different set of links and forms as part of the response of validation (next step affordances) because the flow is different. In this case what is the set of the next links and forms returned from the endpoint? Is it the union of the two workflows and the client knows how to get what it needs? Or does the client send information of which flow it is in and the server uses the information to figure out what response to give it?”

Decoupling Backend Processes from Public URIs
This kind of question comes up frequently. Essentially, there are a couple assumptions here that are worth exploring. The first is the idea that a backend operation (e.g. “validateAddress()”) is exposed over HTTP as a single endpoint, no matter the calling context. This is not a requirement. In fact, it is advantageous to decouple public addresses (URI) from private operations on the server. HTTP (whether using HTTP-CRUD, Hypermedia-REST or some other model) offers the advantage of using multiple public URIs to point to the same backend operation. For example, it is perfectly correct to publish both /validateExistingAddress and /validateNewAddress URIs each of which points to the same  “validateAddress()” operation on the server.

Not Everything Needs a URI
Just because the backend server has an operation such as “validateAddress()” does not mean there has to be a URI associated with that operation. For example, the “user updates his account settings” workflow need not have a direct URI call to “validateAddress()”. Instead, there could be an account settings resource (/account-settings/) that supports the HTTP.PUT method and accepts a body containing (among other things) a modified address. Executing this client-side operation (PUT /account-settings/) passes data to the server and – along with other operations – the server calls the “validateAddress()” operation itself and reports the results to the client.

The same can be done in the case of “user tries to buy a product and enters the shipment address”. This address validation could be a small part of the server-side operation and processing of an HTTP.POST to a /check-out/ resource.

Mapping Actions to URI & Method
In the HTTP-CRUD model, the focus is on using URIs to identify entities and/or operations and using the protocol methods to perform actions. For example, an /addresses/ resource that supports adding (POST), modifying (PUT), removing (DELETE) and retrieving (GET) addresses associated with a context (logged in user, check-out processing etc.) In this case, POSTing or PUTing a resource body to the server allows the server to call the “validateAddress()” operation (among other things) and report results to the client.

Mapping Actions to Hypermedia Controls
In the hypermedia model, actions are described using a hypermedia control such as a link or form. The URI is not important in this model. Instead the control has an identifier (e.g. “validate“), indicates a protocol action (“POST“) and lists state data to include in the payload.

In Siren it might look like this:

"actions": [
 {
     "name": "validate",
     "title": "Validate an Address",
     "method": "POST",
     "href": "...",
     "type": "application/x-www-form-urlencoded",
     "fields": [
           { "name" : "Street", "type" : "text", "value" : "123 Main Street" },
           { "name" : "City",   "type" : "text", "value" : "Byteville"},
           { "name" : "State",  "type" : "text", "value" : "MD" },
           { "name" : "ZIP",    "type" : "text", "value" : "12345"}
     ]
     }
 ]

Note that I didn’t bother to enter a value for the href in this example. It could be any valid URL; I just left it out.

Tracking Workflow Progress Within Messages
Here’s another question from Abiel Woldu’s email:

“The concept of which work flow the client is going through – is it code that should reside in the API code itself or it’s something that sits outside in some other gateway or something?”

When implementing processes over HTTP, it’s wise not to rely on stateful multi-request chains. In other words, don’t expect either the client or server to keep track of where some request belongs in a workflow. Instead, include that information in the request and response bodies themselves. This pattern of including all the important context information with each request and response not only assures that the request can be handled independently (e.g. in a load-balanced cluster), it also helps clients and servers to do work within varying time-spans (e.g. a clients can cache the last request to disk and pick things up a day later). In the REST model, Fielding described this as making messages “self-descriptive”.

For example, there might be a use case that prompts human users to provide quite a lot of information (across various UI tabs) before finally submitting this completed set of work to the server for final validation and processing. One way to support this over HTTP is to allow clients to store “work-in-progress” (WIP) records on the server. As each “tab” (or other UI affordance) is completed, the client app is free to execute a POST or PUT operation with the payload to a URI supplied by the server. The stored data would include a value that indicates how far along in the workflow the user has progressed. This same client app could also recall stored WIP records, inspect the workflow indicator and prompt the user to pick up where she left off. Once all the required elements were supplied, the work could be forwarded for final validation and processing.

Dynamic Workflow via Hypermedia
Finally, in some cases, the series of steps in a workflow might vary greatly at runtime. For example, a service might support a multi-tenant model where each instance of “supply all the details for this work” has different steps or the same steps appear in differing order. The “next step” need not be memorized by the client code. Instead, hypermedia servers can inspect the current server-side configuration, check the current progress by the user and then supply the correct “next step” for this particular instance.

In this way, the client app can support a wide range of workflow details without needing custom code ahead of time (or even downloaded code-on-demand). Instead, the client app only needs to be able to recognize the “next step” link and navigate to that resource.

In Summary
In general, when using HTTP:

1. There is no rule that you must expose internal methods as public URIs
2. You may use more than one URI for the same backend operation
3. In the HTTP-CRUD model, you usually map operations by linking URIs and methods
4. In the hypermedia model, you usually map operations by linking controls and state variables
5. It is best to use “self-descriptive” messages to track workflow progress statelessly
6. The hypermedia model supports dynamic workflow progress using the “next step” link pattern

Thanks to Abiel for his questions and his generous permission for me to use his email and name in this blog post. If you’ve got a question that I haven’t answered online before, feel free to ping me via twitter (@mamund) and fire away.

Since our founding, we have preached the vision that enterprises can open their data and application assets programmatically in a secure way. When we started off, the primary driver for opening up was tighter business integration with partners. Today however, the demand for opening up data and application assets has exploded alongside the growth of mobile, cloud, Big Data and the Internet of Things (IoT).

The idea of organizations as walled-off castles is gone. Mobile is forcing organizations to deliver new business apps to customers and employees beyond the enterprise perimeter. Cloud is redefining how applications are consumed and delivered across a hybridized, extended organization. IoT will upend our notions of outside connectivity and data processing. APIs play a central role in making all this happen. Layer 7 gives customers the confidence to open up via APIs, without compromising security or operational integrity.

For us at Layer 7, security has always been a paramount consideration because our customers are enterprises and enterprises care about security. The CA Technologies acquisition reflects a common point of view on how to deliver new business value in mobility, cloud etc. while protecting the data and applications that are the lifeblood of a today’s enterprise.

CA and Layer 7 both appreciate that the old enterprise security perimeter is disappearing and that the only way to effectively enable online business while protecting information assets is to make identity the new perimeter. We need to focus on managing who gets access to what and what they can do with data once they have that access. Put another way, we need to focus on the identity, data and access that drives modern initiatives around Web, mobile, cloud, social and IoT. Together CA Technologies and Layer 7 Technologies offer enterprises the first truly multi-channel approach to enabling the business while securing its information assets.

Looking into the future, one clearly sees the scope for APIs will increase. IoT will make every formerly detached device connected – all through APIs. Where networking used to be about discrete routers and switches, it is now being transformed, via SDN, into something that is programmable and agile – again, this will be brought to you by APIs. And as for the server and storage infrastructure that underpins the data that drives the Web and mobile, Amazon Web Services has given us a glimpse of the future. As the “Web Services” part of that name suggests, APIs will play a significant role in provisioning in management of the cloud.

As we join CA Technologies, we now have the necessary reach and breadth to make Layer 7 the unassailable leader in the API security and management space. For customers, this means more of what they liked plus the ability to accelerate delivery of our original vision. We’re here to help organizations open up via APIs. And we’re open for business.

We know that connectivity and message-based integration are not new concepts for the financial industry. Enterprise architects in the banking world are masters in the art of connecting old systems with new facades and exposing backend resources through multiple channels. But up until now, a conversation with these professionals would be dominated by concerns rooted in the Services Oriented Architecture world: “How do we prevent service proliferation?”, “How do we secure SOAP conversations?”  In fact, it wasn’t too long ago that the mere mention of an API as a strategic initiative would leave these enterprise architects scratching their heads.

Fast forward to today and many banking technologists are explicitly asking for API Management solutions.  They know the terminology of the space, they know what they want to achieve and they know the pitfalls they wish to avoid. In addition, I’ve been amazed at the depth of knowledge that has emerged among these teams, as enterprise developers have invested their own time to learn from relevant tutorials , articles and – of course – blogs posts. The caricature of the enterprise developer as a SOA dinosaur struggling to understand the “new stuff” is fast becoming a myth.

To be fair, there is still great hesitation within the industry when it comes to opening up data and losing control of the user experience but that isn’t stopping banks from applying good API design practice internally. As we’ve said before, APIs are not simply about reaching  anonymous third-party developers. Indeed, organizations can gain great benefits by applying API Management to the interactions that take place in private for their own apps and partners.  But I’ve been astounded by the handful of architects in the banking world I’ve met who are actively experimenting with public API releases.  It seems that the fear of losing control over data, services and products is beginning to lose out to the perceived value of growing the business through a new channel.

Banks provide a great indicator for the direction of enterprise technology and it certainly seems that the “API thing” has legs within this space. If you are in the enterprise world, make sure you have considered how launching an API program might help your business – because it’s increasingly likely your competitors are already doing so.

One of Brian’s first questions was about the difference between M2M and IoT. The best answer I could give him was actually one I had found through an M2M group on LinkedIn: “I see M2M platforms as mainly enabling vertical integration, as they have historically, of a single capability; where I see IoT as more about horizontal integration of multiple capabilities and resources into a larger system. M2M is about communication, IoT is about integration and interoperability.”

So, whereas M2M feeds data into existing vertical silos, IoT is layered on top horizontally, correlating and integrating data from different silos. A good illustration of this vertical–versus-horizontal distinction was provided in a recent More with Mobile article. The realization that the commercial potential of IoT first and foremost requires a new model of data sharing inspired us to create the Layer 7 Data Lens Solution.

Another question that Brian posed was about the protocols and standards underpinning the M2M/IoT ecosystem. Here is my short list of key protocols (in no particular order):

• IBM MQTT
• OMG’s DDS based on AMQP
• RESTful HTTP
• XMPP
• CoAP
• NanoIP
• SSI

I’d certainly be interested to hear if you had any additions to the list. You’ll find background information about IoT protocols on Telit’s M2M blog and Michael Holdman’s blog. Also, Michael Koster published a very interesting blog post about adding event-driven processing to REST APIs, trying to bridge the necessity of supporting event-driven patterns in IoT within a RESTful API approach.

I’ll be discussing IoT in more detail myself when I take part in Layer 7’s latest API Tech Talk, on Wednesday May 29 at 12pm EDT/9am PDT. If I answer your IoT-related question live during the Tech Talk, Layer 7 will send you a free T-shirt. See you on Wednesday!

The Internet of Things is a simple concept: objects being connected to the Internet. What’s not so simple is managing the enormous, almost sublime amount of data these connected “things” (vehicles, appliances…) generate. There’s also the question of how you give people within your organization secure-but-seamless access to specific subsets of data they can actually make use of.  Well, our man Holger knows how it’s done, so start getting your questions together and join our live Q&A on May 29.

Click here to get the full event details and a reminder in your calendar. On the day of the event, join us at:

•   layer7.com/live

And don’t forget, you can ask questions throughout the stream by chatting or tweeting. Alternatively, you can email your questions in advance and Holger will give you an in-depth answer on the day. IoT is a pretty hot topic right now, so this is bound to be a lively discussion. See you next Wednesday!

Here’s an example of how a business partnership might come about these days. Interaction with an online API partner portal will act as the initial “conversation” that leads to the partnership. If you want to integrate with Salesforce.com, you go to the Salesforce partner portal, figure out the relevant SDK/API, build an app and then submit it to the Salesforce AppExchange.  You don’t ever need to actually talk with anyone at Salesforce to become a business partner with the company.

Another example is the way many companies now enable access to their Web sites via Facebook Connect, Google+ Login or Twitter Login. This represents the first step towards establishing a business partnership with Facebook, Google or Twitter. It’s not new in the Web world and has been discussed for years. What makes it relevant to this discussion is the way it’s being applied to out-dated business processes and practices.

Great platform companies have realized this, “robotized” their business development processes and rationalized their business development teams. As robots are to manufacturing, APIs are to business development. Better technology means that we can focus our human resources on more valuable activities, since handshakes are now being made over OAuth instead of costly dinners and drinks.

“Government information shall be managed as an asset throughout its life cycle to promote interoperability and openness, and, wherever possible and legally permissible, to ensure that data are released to the public in ways that make the data easy to find, accessible, and usable” (emphasis mine).

No Dumping
The usual approach to this type of work is to simply publish raw data in a directory or repository and then create some fencing around the data that helps track usage and distribution. Essentially, making government data “open” becomes a data dumping operation. This practice fails on all of President Obama’s three key points. First, data dumps make finding valuable information not at all easy. Second, even though the content might appear in a standard format like XML, CSV or JSON, it is hardly accessible (except for to geeks, who love this kind of stuff). And finally, raw data is hardly ever usable. Instead, it’s a mind-numbing pile of characters and quote marks that must be massaged and re-interpreted before it comes close to usability.

So, while this new directive offers an opportunity to make available a vast amount of the data the government collects on our behalf, the devil is in the details. And the details are in the interface – the API. As with poorly-designed kitchen appliances and cryptic entertainment center remote controls, when it takes extensive documentation to explain how to use something, the design has failed. There’s a simple principle here. Poor API design results in unusable data.

Affordable Data
It doesn’t have to be this way, of course. Government departments have the opportunity to implement designs that meet the goals set forth in the executive order. They can make it easy for people to find, access and use the data. They can publish not just data but APIs that afford searching, filtering and exploring the data in a meaningful and helpful manner; APIs that empower both users and developers to successfully interact with the data, without resorting to a dashboard featuring dozens of options or mind-numbing explanations.

In the (likely) event that the initial open data release consists of mere data, companies and individuals would be well advised to resist the temptation to build a multitude of “one-off” applications, each of which solves a single problem or answers a narrow set of questions for some subset of the data. Instead, work should be put into converting the raw data into usable API formats such as Atom, OData, HAL, Collection+JSON and HTML (to name just a few). APIs should be designed with the same care that would be given to any interactive experience.  Investment in tools and technologies that can properly represent the data in multiple formats while supporting various use cases and access requirements will yield great results.

Open Data APIs
In the end, organizations that know the importance of a good interface, the power of choice and the freedom of flexible representations will be able to convert raw data into valuable information, which can be consumed by a wide range of users, platforms and devices. These considerations are essential to building and supporting open data APIs.

Because – ultimately – data isn’t open, unless it’s “easy to find, accessible, and usable”.