Customer Stories

Australian Open 2016: Streaming Social Sentiment with Bluemix’s Hybrid Cloud

In the Rod Laver Arena at the Australian Open, the chair umpire announces the conclusion of an exhilarating tennis competition, “Game, Set, Match!” Millions of tennis fans from around the world erupt through social media with a concise message that asserts individual opinions and sentiment.

On a hybrid cloud within IBM’s Continuous Availability Services, Natural Language Technologies are applied to streams of tweets to understand the opinions of the Australian Open fan base. The resulting social sentiment analysis within IBM’s SlamTracker application is regularly updated on the event’s website:

Social analytics within IBM's SlamTracker application displays social sentiment for each professional tennis players

The platform that computes the social sentiments for players is distributed across a hybrid cloud that includes IBM Bluemix, SoftLayer, and the IBM Continuous Availability Services-Events Infrastructure (CAS-EI) as IBM’s Private Cloud. The IBM Continuous Availability Services team delivers 100% uptime with two active sites and one standby site.

The two active sites for both Bluemix and SoftLayer are located in Dallas and London. The standby site for SoftLayer is in Melbourne while parts of the standby Bluemix are in Sydney and London. The monitoring and deployment aspects of the social sentiment application runs within a geographically disperse private cloud.

Social sentiment application is supported by a Continuous Available (CA) geographically distributed architecture

Technology mix behind the social sentiment application

The technology mix that runs the continuously available social sentiment application is a combination of IBM Streams microservices, Object Storage, Unstructured Information Management Architecture (UIMA), private monitoring microservices, encryption technologies, and server load balancers:

Overall continuously available social sentiment technology mix and architecture

Analysis by the Bluemix Streaming Analytics service

The Bluemix Streaming Analytics’ code is developed and built into Stream Application Bundle (SAB) files for deployment onto each Bluemix region. Two streams jobs are involved:

  • Streams job “social1” connects to Twitter and pulls tweets into the streaming application
  • Streams job “social2” applies Latin filters, Unstructured Information Management Architecture (UIMA) Pear files for Natural Language interpretation, window aggregation, parallelizes work, and interfaces with SoftLayer’s ObjectStorage
  • The two jobs “social1″ and social2” connect to each other through a Streams Processing Language graph.

During the Australian Open, the combination of active and standby sites in Dallas, Sydney, and London are continuously available with no cumulative planned or unplanned outage. A standby Bluemix site can be activated if the Dallas or London site is scheduled for a planned outage.

Storage by the SoftLayer’s Object Storage service

The SoftLayer component of the hybrid cloud provides data storage with Object Storage. The geographical locations of SoftLayer Object Storage are selected to match the Bluemix Streaming Analytics geographical locations for performance, accessibility, and availability considerations. The pear file that contains the logic, dictionaries, and sentiment definitions is uploaded to Object Storage for periodic download by the Bluemix Streaming Analytics application. A player comma separated values (CSV) file that contains player names and identification codes is also periodically downloaded by the Bluemix Streaming Analytics application. The output of the Bluemix streams application is in the form of a JavaScript Object Notation (JSON) file and uploaded to a SoftLayer Object Storage container.

An Object Storage Java microservice runs on the CAS-EI private cloud to pull down the latest social sentiment JSON file from Object Storage. The JSON file is parsed by a social leadership board application that depicts overall negative and positive Australian Open tennis player popularity. All of the connection properties including passwords, URLs, containers, user id’s, and etc. used to connect to Object Storage are encrypted by an encryption microservice.

Scalability by private cloud

The CAS-EI private cloud provides Netscalers and Global Server Load Balancers (GSLB) to distribute load across all SoftLayer Object Storage regions. Any access to Object Storage will go through the GSLB’s. The GSLB’s provide detections for HTML error codes with the ability to reroute traffic as required.

Monitoring to assure continuous service

Each component of the continuously available social sentiment service over the hybrid cloud is monitored to avoid any planned or unplanned outages. The monitoring microservice ensures that the Stream’s microservice is running, the RESTful interfaces are accessible, and that each SAB file has been successfully deployed into a healthy state. The social sentiment JSON file is continuously checked for stale data and polls Object Storage availability. If any of the monitoring states fail, an alert is sent through Internet Relay Chat (IRC) for both automatic and manual failover intervention.

Summary

Look for the next installment of this series where we’ll elaborate on the design elements and methodology our team used to measure the social sentiment of the global Australian Open fans. We will depict detailed code snippets of our IBM Streams Application as well as design principles for continuous availability. In addition, we will provide code snippets that depict how to monitor IBM Bluemix Services as well as how to connect and download content from ObjectStorage. For fun and just after the event, we will provide the most popular players and their associated social sentiment during key moments of the Australian Open.

Share this post:

Share on LinkedIn

Add Comment
One Comment

Leave a Reply

Your email address will not be published.Required fields are marked *


Barry Graham

Aaron, thanks for a very interesting explanation of what you’re working on.

Reply
More Customer Stories Stories

Sentry – Using Watson to Help Educators

Sentry is a demonstration web app hosted on Bluemix and built to help elementary through high school teachers better understand how their students feel. Sentry does this by using Watson to identify a learner’s overall emotions and attitude, based on the individual’s written compositions. By analyzing a student’s journal entries or essays, Sentry can alert educators as to whether a student appears to be exhibiting negative, dangerous, or abnormal characteristics.

Australian Open 2016: Hybrid cloud architecture of Bluemix Social Sentiment Application

The Social Sentiment Application was continuously available throughout the Australian Open 2016 with impressive results. The IBM hybrid cloud architecture leveraged the strengths of Bluemix, SoftLayer and IBM's Private Cloud. Docker containers enabled rapid and reusable functions throughout the application. The Software Defined Environment on IBM's Private Cloud leveraged the principles of agile and idempotency. The Bluemix Streaming Analytics microservice enabled realtime processing of hundreds of thousands of tweets. Flexible networking within Bluemix connected multiple clouds together within a secure environment. The Social Sentiment Application just worked!

Accelerating Global Trade – Gliding Eagle and IBM Bluemix

The cognitive, enterprise grade cloud platform isn't just for enterprises. Startups are growing at an unprecedented rate and these same startups consume like the enterprise, build like the enterprise, and become the enterprise. Today, Gliding Eagle's global trade application runs on IBM Bluemix Virtual Servers. This allows them to reach clients across the world with global availability.