Mining the Data behind the API Economy: Cloudant meets Bluemix
gcuomo 060000X9CG Visits (8114)
The notion of compositional business is a byproduct of the API economy and is getting more and more real every day. Cloud development platforms, like IBM Bluemix, quickly demonstrate how new applications can be composed from an ever growing catalog of services and the APIs fronting those services.
Through experimentation, we are learning a great deal about how to perfect the design of these new cloud services and APIs. But one fact that we are learning fast is that you need to pay close attention to the data that your service represents. Enabling the effective mining of Data, will be a key to your success in the API economy. Simply put, the utility and quality of the service is predication on how that service handles its data. And handling the life cycle of data in a cloud is not a trivial matter.
We recently acquired Cloudant, which is an exquisite born-on-the-cloud Data Service. Cloudant can be used to ease the pain of managing application/service centric data in the cloud, and can also introduce new options that can further optimize the utility of your app or service, while making it easier to participate in composition.
This article is for designers of cloud services and is presented in two parts. Part one describes and part two demonstrates an emerging design pattern used for the creation of cloud-based applications and services. Specifically, it shows how a data service can be used to simplify and streamline the creation, consumption and composition of data-centric cloud apps and services.
Let’s break this down and start with a generic Cloud Service.
A Cloud Service. Welcome to the API Economy. We’ve just created a service called the “Green” service. In its simplest form, this service has a public API, enabling access to the service Logic and service Data. For those who are hip to MVC architecture, the Data plays the part of M-Model, the Logic is the C-Controller, and the V-View is the API. The API playing the part of “View” might sound odd, but when you think about it, it is the external representation of the service and provides the external view to consumers.
A Composition. A New Cloud Service or App can be composed by “mashing up” capability from the Red and Green services. The New service exposes a new API that also follows MVC and produces a new “aggregated” view on the data from Red and Green services. In this case, the Red and Green services are the exclusive brokers to their data. As you might imagine, the “brokerage” overhead increases as the data grows. As mentioned, it’s incumbent on the Red and Green service providers to manage the lifecycle of their data. The New service is responsible for managing the lifecycle of the new view on the Red and Green data. The quality of the New service is now a function of it’s own quality, plus the quality of all the services that depends on. If one service is a “weak link”, your service is weak. Well, your welcome to the API economy was fun while it lasted.
But here’s some good news.
A Data Service – is a cloud savvy service that majors in Data. Many folks call such services a DBaaS. A good one, natively supports cloud architecture principles including data isolation to support multi-tenancy, data partitioning and sharding to allow toleration to failure and horizontal scale, and is self-served. Given that Data is the resource that puts the most stress on producing and consuming a cloud service, data services like Cloudant are increasing in popularity. The Cloudant DBaaS provides; Massive Read/Write scalability, Non-Stop data availability, faster development and reduced IT/DBA overhead. With a Cloud Data Service like Cloudant, the Green and Red service provider can delegate their Data responsibilities to Cloudant and instantly gain the aforementioned cloud qualities. Now your long pole has become short, and the time you get back can be focused on differentiating your service. Cloudant is the Xanax (anti-anxiety drug) of the API economy.
But there’s more.
The Cloud Clipboard Pattern – By using Cloudant, as a “data broker”, we can construct an optimized usage pattern where the “middleman” is removed by publishing data and it’s shape to Cloudant, like you do when using a Clipboard in MacOS and Windows. This can be very efficient and can eliminate unnecessary data transfers between Cloud Services. Cloudant supports this pattern with technologies including: message queue, event sourcing and data sync, which it recently introduced intended for syncing data to mobile phones.
The way it works is as follows. The Red and Green services provide an API for Data publish and subscribe via the Data Service. Under the covers the Red and Green Services uses Cloudant’s technologies to support the publish and subscribe actions. The New service uses these APIs to securely subscribe to the data points of interest. The Red and Green services both supply the New services “access keys” which is can use to acquire the data from the Cloudant data service. This ensures the only the New service can access the data in question. Forever more, the New service consumes data directly from Cloudant to obtain the data published by the Red and Green services.
This pattern reduces data movement, which can be an issue for services collecting large amounts of data from, for example, devices and sensors (which is something that Cloudant excels at).
The second part of this article explores an example, which illustrates how to efficiently share and mine data between services using IBM Bluemix and Cloudant.