August 20, 2014 | Written by: THINK Leaders
Categorized: Data | How To | Marketing
Business decisions require confidence in the veracity of the data on which they are based. The challenge is that by 2015, 80 percent of the world’s data will be uncertain.
Data uncertainty arises from many sources and in many ways: as text inconsistencies, ambiguities, sarcasm, deception, technical errors, time delays and the approximations inherent in complex models.
The challenge is how to deal with uncertainty when analyzing big data. How do we represent uncertain data, reduce the uncertainty of data, and reason about that data in a way that allows us to make sound business decisions?
New approaches are emerging to account for uncertainty in data at a giant scale.
Step 1. Establish the base
Every organization has data it can trust. This base data comes from sources you can trust and provides the foundation for making sense of less certain forms of data. A key step in managing uncertain data is establishing and providing a unified view of the base so each department has access to a single, trusted source of data.
Step 2. Gather more data
An instrumented and interconnected world offers an abundance of new data. Most of this new data is low veracity data: it comes from uncertain sources, such as location/geospatial, sensors, social media or video. When handled correctly, however, this data provides context and increases the value of your base data. Start by pulling as much data as possible from a variety of sources.
Step 3. Combine the data
Integrating related data helps maximize context and reduce uncertainty. For example, mobile location data in complex environments such as cities is notoriously imprecise. By combining GPS data with recent social updates and accelerometer data—and correlating that with weather data or past purchases—you gain a better understanding of where your customers are and how likely they are to buy.
Step 4. Extract insights
Analyze and extract insights from all available data. Tools such as natural language processing, customer predictive analytics and social media sentiment each go beyond the basics to extract meaning, surface insights and build context awareness. Be clear about the questions you want to answer so you can employ the right set of analytics for the job.
Step 5. Build models and scale to the organization
This includes building models that incorporate uncertain data. Mathematical approaches such as stochastic reasoning or Monte Carlo simulations take uncertainty into account to yield predictive recommendations. To foster a culture that values data-driven decisions, make these models understandable and accessible across the organization.