A third big data pattern for adapting real time analytics
Chris Nott 100000MPDE Visits (1980)
I have described two patterns for adapting real time analytics on this blog. This third pattern gives a high level view of a further example of how adaptive real time analytics has been deployed to realise value. The pattern shows how an analytical model used to continuously analyse incoming information can be updated based upon changing business circumstances.
I shall assume that IBM's InfoSphere Streams is used to implement real time analytics on data in motion and that InfoSphere BigInsights – IBM's commercial offering with Hadoop – holds data at rest.
In this use case, an statistical model is already deployed in Streams to monitor what is happening in some aspect of the business, and that analytics exist in BigInsights to build an updated model. The outline steps of the use case are then as follows:
As with the previous patterns, the implementation of this use case assumes that data reaches BigInsights via Streams. (Data filtered by Streams to store in BigInsights must be sufficient to enable new statistical models to be built.) It means that the data is aligned across both technologies.
Control of the new model building mechanism was implemented in Streams.
One limitation of the pattern is that whilst the model is being updated, analysis of incoming data is no longer optimised for current business need. This may be problematic in fast changing environments.
The main benefit of this pattern is that a business can sustain optimised analytics on incoming data as circumstances change.