thousands of tweets about football matches in real-time. The final stage in the pipeline, the tweet processing microservice, stores the retrieved tweets into a Cloudant database. This post outlines how the Match Tracker uses CouchDB views and change monitoring.
In this blog post, we’ll look at the backend architecture which allows Match Tracker to analyze thousands of tweets about football matches in “real-time”. The application must be able to handle tracking which games are being played, continuous searching for all messages about matches and processing live all the data being produced. Rather than designing a single monolithic application architecture, with all these components and services running as a single unit, we’re going to use the microservices approach.
In this series of blog posts we’re going to walk through building a scalable architecture for processing “real-time” Twitter streams. Using IBM Bluemix, IBM Insights For Twitter, Apache Kafka and Cloudant, we’ll build a processing pipeline using a series of microservices rather than a monolithic application. We’ll look at designing the architecture to support scaling automatically on response to fluctuating load and how to handle failures without losing work.
Last month, a colleague was explaining he was not looking forward to an afternoon of long-distance conference calls. Having recently started using Slack for collaboration with their remote team, they lamented "I wish I could do my conference calls using Slack!" This got us thinking.