It's no secret that data volumes and data sources are proliferating. If you think back to even as little as 12 months ago, you probably have more data sources and volumes now than you did even that short time ago. These data types are varied -- more structured data collected from your systems; third party sources as a service on the cloud; text or data born on the cloud and more. The traditional data warehouse excels at core reporting and analytics, yet it probably is not the most cost-effective way to store every type of data. In the case of text, your storage choices are more limited to solutions such as Hadoop.
Right workload on the right platform: hybrid gets you there
A hybrid data architecture comes in to help you store all of this varied data based on the importance of the data, privacy and governance requirements for the data, the original source of the data and available capacity on your systems. Such an architecture suits varied data placement strategy by handling data on-premises and in the cloud, or both, and it enables management of a full range of data: structured and unstructured, at rest and in motion. These traditional and next-generation technologies are designed to meet a broadened range of analytics or transaction processing needs.
Query across the platform and even to other stores
IBM recognizes the need to store all types of data and has developed a hybrid solution so you can place the right workload on the right platform for each need. IBM Fluid Query (in PureData System for Analytics, dashDB Local and more platforms coming soon) and IBM Big SQL (in IBM BigInsights), provide the ability to query data where it resides and without moving it around just for the purpose of analysis. This integrated capability can query remote data in platforms in the IBM data architecture as well as popular sources including Oracle, Hortonworks and others. For example, PureData System for Analytics can query into BigInsights, DB2, dashDB, Oralce databases or others. dashDB Local can query into PureData System for Analytics, BigInsights or other platforms.
An example: get deeper insights
Here is an example of how your data may be stored and why you gain deeper insights when you are querying more data:
- PureData System for Analytics data warehouse appliance contains sales data
- dashDB Local contains sales person compensation data
- dashDB as a managed service contains customer loyalty data
- IBM BigInsights contains text from social media
- Weather data is available as a service on IBM Bluemix
If you combine all of this data, you can tell how weather affects sales of your products and which sellers have the most success in doing this and which products are selling. By looking at social media, you can tell the need this product is solving under weather conditions and whether or not a trend is taking hold. By using IBM Fluid Query or IBM Big SQL, you can query all of these data sources together to get this deep insight. You have a choice of which platform to use to begin the query -- and often it is recommended to use the platform that contains the most data or the most important data. It is a common SQL engine -- and you are not required to move all of this data around just to satisfy the needs of this one query.
If you query these sources individually, you do not get the same rich insights, but rather simple pieces of a picture. When you combine all of this data, the picture of what is happening becomes richer and more actionable.
In summary, most organizations require a hybrid data architecture to handle the breadth of data coming at them. When you add a common analytics engine, suddenly you can take advantage of value from data. This is richer, deeper insights that provide a business advantage in some way. Now that you understand the hybrid data architecture, use this community to explore and learn about its member products.