Originally published at:http://www.apiful.io/intro/2016/04/26/where-is-the-spec.html

The number and diversity of web APIs available nowadays is, both, already impressive and increasing. Consequently, there are many proposals on how to formalize web API description in machine-understandable ways. Doing so promises means for automation and increased ease-of-use. Nonetheless, formal web API descriptions for public APIs remain a rarity. Why?

APIs are everywhere...

By now, companies small and large expose web APIs, and APIs have become integral part of many business models - consider MailChimp's Mandrill transactional email API or the Mapbox API. For developers, the sheer amount and diversity of web APIs holds a lot of potential. Applications can now make use of many otherwise hard to implement or even inaccessible resources and functionalities. The usage of APIs, however, requires quite some manual effort: APIs need to be identified, their functionalities, qualities, and terms of service need to be assessed, their endpoints and usage requirements need to be understood, and the interaction with an API needs be implemented and integrated with the application. In addition, custom logic may be needed in place in case APIs become (temporarily) unavailable.

...formal descriptions not so much.

Formal web API descriptions aim to remove a large chunk of these efforts. As laid out in this blog before, there are many competing formats like the Open API specification (formerly known as Swagger), RAML, or API blueprints. However, when it comes to public APIs, such specifications are like aliens. Given the huge dimensions of the (API) universe, they should exist but barely anybody claims to have laid eyes on them yet.¹ And this is true no matter which specific format you are looking for. This circumstance begs the question: why? Why don't API providers make formal API specifications available to developers?

Possible explanations

One reason to explain the sparsity of formal API descriptions may be the plethora of competing formats. Providers may be unwilling to bet on any of the competing solutions as long as it isn't clear which one or even whether one will prevail. While this situation may not be ideal, one can automatically translate between existing formats to a large extent. Furthermore, the broad industry support behind efforts like the Open API Initiative and its specification, the evolution of which is designed as an open process, should be reassuring.

Maybe formal web API descriptions are not expressive enough to fit all diverse interaction models that APIs can feature? After all, as I argued in this blog before, flexibility is a major cornerstone of the success of web APIs. However, if the attempts of third parties to generate web API descriptions show something, its that such efforts can succeed for various APIs, the many Google APIs or Facebook's Graph API.

Another possible explanation for the scarcity of formal API descriptions is that they do not provide enough value to justify the efforts needed for their creation. Considering the value part of this argument, I disagree. For API consumers, the value would be immediate. Imagine developers were able to obtain the latest formal description for an API from a consistent source. This would allow to diff versions to detect changes which may impact an application's proper functioning. Even better, client-libraries can automatically be derived (and kept up to date) in any desirable language from such descriptions. API providers, on the other hand, can use formal API descriptions to document their APIs in- and externally, and they can provide their consumers with a plethora of development tools without having to implement them. Both, providers and consumers can profit from automatic testing and monitoring build around formal API specifications. So, if formal API descriptions provide enough value, maybe the effort required to create and maintain them is too high? In my opinion, this rings true, especially given modern agile development methods that do not start with a specification (as in a classical waterfall model). Rather, in many cases, specifications become an afterthought and their need arises first when it comes to exposing an API to partners, or the external world in general. Furthermore, as we learn from projects within IBM, web API specifications easily grow into the thousands of lines. Asking developers to maintain them manually is asking a lot.

Change for the better

In our group, we are working on ways to improve the automatic generation and maintenance of web API specifications to address this issue. This requires, among other things, to find the right trade-off between enabling API providers to influence the automatic generation enough for the resulting specification to be correct and meaningful while at the same time not demanding so much input as to be deterrent.

What do you see as the reasons for the lack of API specifications made available by providers?

What do you think would improve the situation?

1: To refine this statement: there are some efforts to provide formal API descriptions here and there, but they mostly rely on third-parties, who either manually create them or build scraping tools to do so.