Cognitive Computing

Building cognitive IoT solutions using data assimilation

Share this post:

How can we predict, with certainty, the motion or direction of clouds in the sky? How can we track an oil spill in the ocean, locate a gas leak in a city, or identify an arterial blockage in our bodies? Data assimilation methods can be used across a range of cognitive IoT applications to do that and much more.

Data assimilation is the backbone of modern forecasting systems, which link real world data with mathematical models. Data assimilation methods improve the accuracy of forecasts provided by models by optimally combining sensor data with model states, and evaluate forecast reliability by taking into account possible uncertainty, i.e., model errors or sensor noise.

Mathematically, data assimilation relies upon filtering theory which provides a best estimate of the true state of a model. Up to now, most of my work has focused around developing new minimax filtering algorithms for dynamical systems arising in geophysics (weather, rivers), urban environments (traffic, gas leaks) and biology (blood flows). Minimax filtering quantifies uncertainty in the predicted state of a system by establishing minimum and maximum limits for errors. In essence, creating a set wherein the true state of the system lies. The work on minimax filters has been done in close collaboration with academia (Utrecht University, INRIA, UC Berkley, University College Dublin) and IBM researchers from Dublin, Melbourne, Zurich and Yorktown labs.

Currently I am working on cognitive filtering, a fusion of ideas from machine learning, numerical analysis and control theory which is expected to work for massive unstructured data streams like Twitter data.

The most recent application of minimax filters is a new algorithm for solar energy forecasting which came out of a fruitful collaboration between IBM Research labs in Yorktown and Dublin. Solar energy is the most abundant form of renewable energy resources and its contribution towards the total energy mix is rapidly increasing. However, integration of high penetration level of solar energy in the electric grid poses significant challenge and cost. The cloud movement, formation, dissipation and associated variable shading of solar panels may result in steep ramps of solar power being injected into the grid. The variability and uncertainty of solar power often forces the system operators to hold extra reserves of conventional power generation, which adds cost. Ongoing research as well as previous experience of wind power integration shows that accurate solar forecasting plays a key role in the reliable and cost-effective integration of solar power.

Our algorithm uses Cloud Optical Depth (COD) images provided by a satellite to forecast cloud motion. Conventionally, cloud motion is forecasted by using wind fields generated by numerical weather prediction models. It turns out, however, that inaccuracy in the estimate of the altitude of the clouds may still lead to significant prediction error even if the weather model’s wind fields are quite accurate. To overcome this issue, we developed a simple yet powerful approach using state-of-the-art image processing tools to estimate an “optical flow” from two consecutive COD images. These flows are then used to predict cloud motion by utilizing a minimax filtering technique. The algorithm has been successfully validated on COD images provided by two geostationary operational environmental satellites from NOAA serving the Western Hemisphere.

ANGIOGRAM (displaying the motion of the dye within an artery)

ANGIOGRAM (displaying the motion of the dye within an artery)

Along with colleagues in IBM’s Melbourne and Almaden labs, we are applying data assimilation and filtering techniques to the dynamics of blood flow in the body. We are researching how computer simulation can be used to replace catheter based Fractional Flow Reserve (FFR), a technique used in catheterization to measure pressure differences across a stenosis (a narrowing of a blood vessel).

Stenosis (In Yellow)

STENOSIS (In Yellow)

To localize the stenosis and to estimate how severe it is, image processing could be used to build 3D computer models of coronary arteries from a sequence of angiograms. This model is then used to simulate blood flow in order to match the speed of the simulated flow to the data: the speed of the blood flow, extracted from angiograms.

The “matched” blood flow is then used to estimate the pressure drop around the stenosis region. Our data assimilation method optimally estimates blood pressure from the data that allows us to quantify stenosis hemodynamic significance (i.e. how it impacts blood supply). If the pressure drop is severe, a doctor will recommend a stent be surgically inserted.

The Matched Blow Flow (Bottom)

The Matched Blood Flow

These projects are just two examples of how IBM Research is exploring potential applications for data assimilation and cognitive filtering techniques. At its core, data assimilation is about the whole process of having an idea, formalizing it by writing down equations of a mathematical model, implementing the model and, finally, making it come alive by pulling in the data.


  1. S. Zhuk, J. Frank, I. Herlin, and R. Shorten. “Data assimilation for linear parabolic equations: minimax projection method”. In: SIAM J. Sci. Comp., 2015
  2. S. Zhuk; M. Petreczky, “Minimax Observers for Linear Differential-Algebraic Equations,” in IEEE Transactions on Automatic Control, 2016
  3. S. Zhuk; A. Polyakov; O. Nakonechnyi, “Note on Minimax Sliding Mode Control Design for Linear Systems,” in IEEE Transactions on Automatic Control, 2016
  4. S. Zhuk and T. Tchrakian. “Parameter estimation for Euler equations with uncertain inputs”. In: Proc. of IEEE Conference on Decision and Control. 2015.
  5. T. Tchrakian and S. Zhuk. “A macroscopic traffic data assimilation framework based on Fourier- Galerkin method and Minimax Estimation”. In: IEEE Transactions on Intelligent Transp. Systems, 2014, (special issue).
  6. S. Tirupathi, T. Tchrakian,S. Zhuk and S. McKenna. “Shock capturing data assimilation algo- rithm for 1D shallow water equations”. In: Advances in Water Resources, 2016.
  7. S. Zhuk, T. Tchrakian, R. Shorten and S. Moore. “On Source-Term Parameter Estimation for Linear Advection-Diffusion Equations with Uncertain Coefficients”. In: SIAM J. Sci. Comp., 2016.
  8. A. Akhriev, H. Hamann, S. Lu, T.  Tchrakian and S. Zhuk,. “Where computer vision can aid physics: dynamic cloud forecasting from satellite images”, CVPR17, (submitted)
  9. Moore , S. Zhuk et al., Real-Time Cloud-Based Virtual Fractional Flow Reserve Estimation, Patent application (filed, 2016)


More stories

AI helps explain your microbiome

Newly published research describes an Explainable AI to help understand the link between skin microbiome composition and personal wellbeing.

Continue reading

Moving beyond the self-reported scale: Objectively measuring chronic pain with AI

Together with Boston Scientific, we are presenting research that details the feasibility and progress towards our new pain measurement method at the 2021 North American Neuromodulation Society Annual Meeting.

Continue reading

COVID-19 a year later: What have we learned?

We’ve learned a lot during the past year about how to address global crises, but in my mind, one lesson cannot be ignored: The need for more strategic collaborations across institutions and sectors.

Continue reading