The convergence of operational and analytical data

Share this post:

As expected in the first quarter of the year, there are many “top trends for 2021”. This year, I am reading these with a greater focus than before looking for clues of what may happen post this pandemic.
Although I have seen some interesting predictions, like

• Hyper / Full automation using for example neuromorphic computing, emulating or mimicking the neural structure and operation of the human brain.
• The inseparability of technology and business, where the CIO will often be the COO by proxy.
• The increased importance in the “voice of the society”, where internal and external content and data are increasingly getting mixed and require continuous management

(Source: Gartner, Strategic Technology Predictions for 2021 by Daryl Plummer.*)

Much of my own thoughts are going towards a long-time trend, not necessarily being triggered by the pandemic. Although I can see some indirect influence of the current situation, such as a stronger focus on AI and automation to optimize online sales.  But what I am thinking of is the convergence of operational and analytical use of data.

Operational and analytical synergies

The world of data used to be divided between the applications and processes creating and updating data and the solutions and processes analyzing data. The team that worked with the operational applications did not talk to the team that worked with the analytical solutions.
Very rarely did the two meet. It was usually a one-way scheduled flow of data from operational applications to analytical solutions.
With businesses increasingly being digitized and the growing interest in using analytics and AI to optimize and automize operational processes the two worlds need to become one.

Like any relationship there is compromises needed to make both sides flourish and feel comfortable. But the phrase “one plus one equals three” fits very well, I think.

There are many related trends that is worthwhile looking at in this context, like DevOps, Lambda Architecture, Two-speed IT, Agile etc.
But the key things to look at when it comes to operational and analytical data convergence are:

• Event-driven, capturing, analyzing and acting on data when events occur. Rather than after the event, based on a pre-defined schedule.
• Bi-directional, where needed, the data needs to be able to go both from source to target and back from target to source, rather than being limited to one-way.
• Micro services and API-based – loosely coupled architecture, allowing for faster changes and modular design, avoiding the limitations of large monolithic solutions.

Data and analytics architecture considerations

If I connect this to the 2021 predictions I referenced above, then three important architectural considerations appear to me:

1. Automation. As the ambition for automation goes towards “Hyper / Full automation”, the requirement on data becomes challenging. Especially in terms of latency (the freshness of data). This is why it is important to have an event-driven architecture and design. An example would be an automated real time sales workflow. Where it is crucial to make the data available to the Analytical / AI algorithm that serves as the basis for automatic decision making at a certain point in the online sales workflow. In a previous article I have elaborated further on making data “fit for purpose”:
   https://www.linkedin.com/pulse/what-does-take-make-data-fit-purpose-tommie-hallin
2. Operational. Data & Analytics are becoming increasingly crucial for business operations. As data and analytics become mission critical, bi-directional workflows, the management and organizational set up also needs to step up and have appropriate services in place. Which relates to the role of a CIO and COO progressively dissolve into one.
3. Responsive. The importance of being able to quickly react to external events (as described in the “voice of the society” prediction), and changes in the market requires a modular and agile architecture. By building data & analytics solutions as a set of loosely coupled components, using microservices connected via API’s the impact of the changes is easier contained and thereby limiting both the effort and time to make them available.

Transforming the data and analytics architecture

Some of my IBM colleagues wrote a paper that goes into a similar direction, expanding into the data challenges and opportunities of multicloud, DataOps and “containerization”. Have a look:

https://www.ibm.com/thought-leadership/institute-business-value/report/multicloud-data-strategy

I think that 2021 will be the year when many organisations take important steps towards transforming and orienting their data & analytics architecture for the convergence of operational and analytical use of data. And I feel very inspired being part of this long coming transformation.

As always, please share your thoughts on this.