About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Improving ML engineering with data observability
The Weather Company + IBM Databand
Qaish Kanchwala is a Machine Learning (ML) Engineering Manager at The Weather Company®. He manages a team of eight engineers, including DevOps, ML and data engineers. They’re responsible for building and training the ML models used in production for The Weather Company. Most of his responsibilities involve designing solutions for the engineering team and making sure the work gets done on time.
The Weather Company has moved toward being a data-first organization. For Kanchwala’s team, this means working with data on ML use cases for customer advertising, personalization and health conditions predictions. Since the future state of advertising no longer relies on cookies or other identifiers, his team uses data to predict user segments. These user segments are then used for various advertising campaigns.
The accuracy of these user segments can have an impact on revenue generation, so it’s critical that Kanchwala and his team are using the most accurate data, optimized for these campaigns. For example, less accuracy in the models could result in an advertising campaign that under-indexes to the segment the customer aims to reach or that does not reach the intended audience segment.
Since they use data pipelines such as Apache Airflow and Sagemaker to make these model predictions, the pipelines need to be reliable, and the data needs to be accurate.
“For our perspective, a lot of business decisions are being made on the segments and predictions that we make,” says Kanchwala. “As we built these segments, we strive to ensure that the data going into the prediction pipelines are accurate so that the predictions coming out of those pipelines are accurate. Any loss of accuracy here could impact someone’s business decisions or bottom line.”
Like for most data and ML engineering teams, it was challenging to track model performance over time and input proactive alerting to be notified when changes occur. If his team is unaware of data issues, then a customer could be making decisions using predictions based on outdated or less relevant data.
These challenges led The Weather Company to implement IBM® Databand® software as its data observability solution. Databand helps the company proactively resolve data issues before they may impact the business.
Before Databand, Kanchwala’s team lacked a complete monitoring tool to track data drift over time. The limited number of alerts and reports they did have required a lot of manual intervention.
“We have looked into using other tools, but at the end of the day they didn’t fit into our data engineering process for lineage,” says Kanchwala. “Other tools might be great for application or memory monitoring but not for data pipelines.”
The team uses Databand’s “always-on” data monitoring capabilities to track data drift overtime for their ML features and model outputs. From a data engineering perspective, Databand shows data pipeline lineage and the impact analysis during run-time.
Click here
Since using Databand, the data and ML engineering team improved their data lineage and SLA tracking.
“Without an operational view such as Databand’s, it would be extremely hard to understand the overall health of our ML pipelines,” says Kanchwala. “The integration of availability tracking and aggregated metrics from Airflow has been super useful. I appreciate looking at Databand and seeing the Airflow data within one dashboard.”
Overall, The Weather Company has improved its data engineering KPIs with:
- Continuous visibility and transparency: Databand’s operational view instantly shows the health of its Apache Airflow and Sagemaker pipelines.
- Improved SLA alerting and metrics tracking: The Weather Company has implemented Databand as a “quality gate” before pushing changes to production. This forces data and ML engineers to perform a mandatory quality check in development before pushing ahead into production.
- Data quality monitoring: Since Databand integrates with any Apache Airflow environments, data engineers can see exactly which step causes a data incident and resolve it quicker.
Click here
The Weather Company is the world’s leading weather provider1, helping people and businesses make more informed decisions and take action in the face of weather. The Weather Company’s high-volume weather data, insights, advertising and media solutions across the open web help people, businesses and brands around the world prepare for and harness the power of weather in a scalable, privacy-forward way.
1 According to Comscore, The Weather Channel was the largest provider of weather forecasts worldwide (web and app) in 2022 based on the average of the total monthly unique visitors. Comscore Media Metrix®, Worldwide Rollup Media Trend, News/Information – Weather category incl. The [M] Weather Channel, The, Jan-Dec. 2022 avg
Legal
© Copyright IBM Corporation 2023. IBM Corporation, New Orchard Road, Armonk, NY 10504
Produced in the United States of America, December 2023.
IBM, the IBM logo, ibm.com, and Databand are trademarks or registered trademarks of International Business Machines Corporation, in the United States and/or other countries. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on ibm.com/legal/copyright-trademark.
Autobrand®, Cloud and Rainbow™ device, Icebreaker Studios®, Social®and device, The Lift™, The Weather Company®, The Weather Company®and device, The Weather Underground®, TWC®, Weather Bonk®, Weather Exchange®, Weather FX and device™, Weather Means Business®, Weather Quickie®, Weather Underground®, Weather.com®, WeatherFX®, WU®, WU® and device, Wunderground™, Wunderground.com®, Wundermap®, and Wunderradio® are trademarks or registered trademarks of TWC Product and Technology, LLC, an IBM Company.
This document is current as of the initial date of publication and may be changed by IBM at any time. Not all offerings are available in every country in which IBM operates.
All client examples cited or described are presented as illustrations of the manner in which some clients have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations and conditions. Generally expected results cannot be provided as each client's results will depend entirely on the client’s systems and services ordered. THE INFORMATION IN THIS DOCUMENT IS PROVIDED "AS IS" WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING WITHOUT ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OR CONDITION OF NON-INFRINGEMENT. IBM products are warranted according to the terms and conditions of the agreements under which they are provided. [