Community

Actionable Compression in the Palm of Your Hand: Using Ruby to Put Analytics in a Mobile App

Share this post:

Every day 2.5 exabytes (2.5×1018) of data are created which means 90% of the world’s data was created in the last two years alone.  Data production rates are only increasing with the volume of business data created every year doubling every 1.2 years.  That means by 2020 data production rates will be 44x what they were in 2009.  This is a staggering amount of data is captured in every growing clouds, SANS, and harddrives, but it goes mostly un-mined. Tapping into this resource through analytics gives us new ways to understand customers, deliver products and services and respond to change.  

The challenge with analytics has been two fold. First, the computational load resulted in processing subsets of data with tremendous hardware overheads.  Secondly, it took a data scientist weeks, months, or years to give you useful analysis.  The problem is you spent a lot of money to be out of sync with the backlog of data that continued to stream into the system.  The compute power and time to analysis meant that “real time” analytics were often not so real time.  That’s why the Bluemix team was excited to see a demonstration of IBM Blu Acceleration technology that made it possible to do on-the-fly analytics surfaced on a mobile application.

Blu Acceleration delivers analytics faster than ever

  • In memory data is no longer a limit:  In the past, data sets were limited to what could be held in RAM. With Blu Acceleration that limit does not exist which means data sets can be truly massive.
  • Actionable Compression: Tools like offset coding are very useful with numeric data; instead of trying to compress the values 100, 101, 102, and 103, for example, DB2 will store a single value (100) and just the offsets to that value (1, 2, 3, etc.).  What it means is that we can compress the data improving our I/O stream gaining 35x to 73x faster analytics.
  • Data Skipping: This is what IBM Blu Acceleration does to deliver an order of magnitude (10x) savings across CPU, RAM, and I/O.  Data skipping enables DB2 to detect ranges of column values that are not needed to satisfy a query and avoid reading pages containing those values from disk.
  • CPU Acceleration: BLU Acceleration takes advantage of single instruction multiple data (SIMD) processing, if it is available on the hardware being used. By exploiting SIMD instructions, which are very low-level specific CPU instructions, BLU Acceleration can perform the same operation on multiple points of data simultaneously.

In a recent webcast on mobile app development webcast with Mario Briggs and Praveen Devarao demonstrated how to put IBM Blu Acceleration technology into a mobile application.  You can also read their DeveloperWorks Article Create a business intelligence and analytics service in Ruby with Analytics Warehouse on IBM Bluemix.  There is demo code available for developers to tinker with and use. 

Don’t have a Bluemix account? Get started building for free.

More stories
May 7, 2019

We’ve Moved! The IBM Cloud Blog Has a New URL

In an effort better integrate the IBM Cloud Blog with the IBM Cloud web experience, we have migrated the blog to a new URL: www.ibm.com/cloud/blog.

Continue reading

April 19, 2019

Reach Out to the IBM Cloud Development Teams on Slack

Get the help you need fast—directly from the IBM Cloud Development Teams and other users on Slack.

Continue reading

April 11, 2019

Permanent Redirect to cloud.ibm.com from console.bluemix.net

Starting on April 27, 2019, we will be turning on permanent redirects from bluemix.net to cloud.ibm.com. All of the same functionality that existed on bluemix.net is still available in cloud.ibm.com.

Continue reading