January 10th, 2013

Measuring Hackathon ROI for APIs

Hackathon ROII often get asked whether hackathons actually provide API publishers with any true, measurable return on investment (ROI). The simple answer is “yes” – and the positive benefits of hackathons are now undeniable.  However, the benefits can be a little hard to quantify, making ROI tricky to measure objectively.

For example, hackathons provide a fantastic way to grow developer awareness of your API as a brand in and of itself, separate from your core business. When the developers who attend your hackathon go back to their day jobs on Monday, they have added your API to their programming tool belts and will use it, when appropriate, in upcoming projects. Additionally, hackathons will attract the attention of thought leaders and influencers who will mention your API on blogs and forums, spreading the word further. These benefits can deliver considerable value but they can also be difficult to quickly quantify.

Nevertheless, API evangelists will be held accountable for demonstrating the real-world value of their hackathons. One way to do this is to show how hackathons enable your company to conduct developer user experience (DevUX) research at a minimal cost. Gathering feedback and data from hackathons provides the most cost-effective way to optimize the quality of your API as a product by answering questions like:

  • How user-friendly is my registration process?
  • Do my APIs ever return incorrect or unexpected results?
  • What new features should I add to future versions of my API?
  • Is the skill level of my API appropriate for long-tail app developers?
  • What kind of tutorials and other documentation will my developers need?
  • Which programming languages are my developers using to implement my APIs?
  • How useful is my API and what are the most common/innovative use cases for it?

The data and feedback you gather will also help you to further demonstrate ROI by providing the answers to questions such as:

  • How many developers registered and how many actually attended?
  • Did the hackathon appeal to the types of developer we want to attract?
  • Did any valuable or innovative apps get prototyped?

Hackathons offer a fantastic way to build excitement around your API and optimize the quality of your interface. If you still have any doubts, join us for a hackathon (and participate!) to see how other API platforms are doing it.

3 Comments »

  1. I would expand on the “How user-friendly was my registration process?” question to: what were the major barriers for the developer getting started with our API and subsequently solving the problem they set out to solve? I you see developers stumble repeatedly over things like wacky auth schemes or no examples to start from, those are giant flashing red flags for you to pay attention to because guaranteed the rest of your customers are running into the same issues.

    Comment by John Sheehan — January 10, 2013 @ 9:30 pm

  2. As far as ROI goes, the best thing you can do at a hackathon is build a relationship and become a developer’s ally going forward, even if that doesn’t directly or immediately involve your product. I cannot tell you how many times just being generally helpful and prioritizing helping developers be successful paid off for us in the long run at Twilio. In really big ways sometimes 6, 12, 18 months later.

    Comment by John Sheehan — January 10, 2013 @ 9:32 pm

  3. John thanks for your comments…

    re: major barriers vs “user-friendly” –> totally correct, I took an API to a hackathon a while back and the Auth was so complex (and overly secured) that not even the most experienced iOS rockstar dev could get it working in the course of 24 hours. as the evangelist I had only tested making sample calls to their sandbox API (it was a brand new API) and not bothered w/ the actual auth yet, so it was a lesson learned for me as the evangelist. doh!

    re: ROI and becoming a dev’s ally – yes – be sincere to the devs, which is proven through actions over time. also reinforces the importance of having a technically savvy evangelist, basically a support tech by day, who is truly helping the developers build their apps. but also maintaining that over the long term, helping them get their app published, supporting them when they get discouraged, etc. I always like to see a tab in an API Portal for featured apps where the API platform promotes the devs. Some API publishers even go so far as to blog about the apps, and get their PR dept involved to promote even more.

    Comment by Alex Gaber — January 11, 2013 @ 10:44 am

RSS feed for comments on this post. TrackBack URL

Leave a comment