Clients around the world rely on the Met Office to provide accurate weather and climate data. How could the organization enhance its ability to share forecasts while controlling costs?
The Met Office adopted a hybrid cloud infrastructure, using IBM Spectrum Scale™ to push data from an on-premises cluster based on IBM flash storage and IBM Z® solutions to a public cloud platform.
200,000 GBPspend avoided through storage compression
Speedssharing of accurate forecasts, helping the UK and other economies prosper
>300 millionweather observations ingested per day
Business challenge story
Recognizing the true impact of the weather
Every day, the weather and climate touch all of our lives in multiple ways. For businesses, access to accurate forecasts can be a game-changing differentiator. The UK’s national weather service, the Met Office, works around the clock with its partners to make reliable weather and climate advice available to all.
Peter Chase, zLinux & Storage Specialist at the Met Office, says: “Our forecasts are used in diverse ways by clients across industries. For example, highway agencies and airports need to know when it’s going to be icy, so they can grit roads and prepare equipment. Supermarkets want to know when they can expect a cold snap or a heatwave, so they can stock up on tissues or sunscreen. Our data is used by scientific researchers to lead the way in climate change studies, by government agencies choosing where to place wind turbines – and that’s just the start.”
A forecast is only a forecast if it is delivered in advance of an event, and the longer that organizations have to react to predictions, the better they can respond. This makes it essential that the Met Office provide weather data to clients rapidly, precisely and reliably.
“The accuracy of our forecasts depends on building models using as much observational data as possible,” explains Chase. “They’re only useful if we can deliver them on time, whenever they’re needed. To meet these demands, driving down the latency of our infrastructure is an ongoing priority. At the same time, we’re always looking to find new efficiencies. We saw an opportunity to address data sprawl by moving to a hybrid cloud model, which would simplify data sharing and reduce costs.”
Doing more with data
As part of a ‘cloud-first’ IT strategy, the Met Office decided to deploy a hybrid cloud solution that connects high-performance on-premises infrastructure to a public cloud platform: Amazon Web Services, where it can provide data to clients in a consumable format.
On-premises, the Met Office implemented a stretched cluster consisting of an IBM z13® server and IBM all-flash storage at each of its two production sites, with data replicated synchronously between them. Using IBM Spectrum Scale, the Met Office manages data across the hybrid cloud environment.
“For data-intensive processes that need high uptime and stability, we think of IBM Z first,” comments Chase. “We wanted very cost-effective storage, which we thought might rule out flash options. However, the IBM Spectrum Virtualize technology gives us a 2:1 compression ratio for data so we can buy less capacity, giving us both exceptional performance and efficiency. Spectrum Scale was the natural choice for the cluster file system, as it enables us to automate movement of huge amounts of data between our different systems with a high degree of control.”
The cluster acts as a transient data hub, with a data residency policy of 48 hours or less. It ingests more than 300 million weather observations per day, making them available for analysis in the Met Office’s supercomputing environment. It receives 10 TB of results from the supercomputer daily, and presents this to downstream systems hosted in the public cloud. For development, the Met Office runs parallel versions of data on the cluster, equating to an additional 7 TB per day.
“We’re deploying IBM HyperSwap, an IBM Spectrum Virtualize feature, to provide even higher availability for the environment,” says Chase. “Choosing solutions from a single vendor means there isn’t any finger-pointing if we run into problems – we can rely on IBM to help us make the most of the technology.”
Forecasting the weather, come rain or shine
The Met Office is making significant cost savings through data compression on IBM all flash storage. It is also increasing efficiency by reducing the management load on its IT team.
“We achieved a compression ratio of 2:1, which we estimate enabled us to avoid spending approximately 200,000 GBP on storage,” remarks Chase. “When you consider that we haven’t compromised on performance at all – in fact we’ve increased it considerably – these savings are even more impressive. The IBM solutions are designed to be extremely easy to manage and include automation tools. As a result, a smaller team can administer the environment even as it expands and gets more sophisticated.”
The new architecture decreases latency and boosts availability, helping the Met Office build more accurate forecasts and deliver them to clients more quickly. Equipped with timely, precise forecasts, users can prepare more effectively for changes in weather and climate and the Met Office can increase its level of service to the UK and other economies.
“IBM solutions slash our response times, and we’re seeing the impact internally and externally,” says Chase. “Our data scientists get the latest observations earlier, while our clients get the forecasts they need for better decision-making sooner.”
Deploying IBM solutions is helping the Met Office realize its cloud strategy, serving up its data as a basis for innovation.
Chase concludes: “Working with IBM, we continue to make our data easier to share so that users can build new products and services that are more attuned to changes to in the weather and climate than ever before. The combination of IBM Z, all flash storage, and Spectrum Scale solutions give us confidence that we’re always providing a single version of the truth based on the latest and greatest data.”
The Met Office is the UK's national meteorological service. Employing more than 1,700 people at 60 locations around the world, it is a global provider of weather and climate services. The Met Office is recognized as one of the world's most accurate forecasters, using more than 10 million weather observations a day, an advanced atmospheric model and a high-performance supercomputer to create 3,000 tailored forecasts and briefings a day.
Take the next step
To learn more about IBM Flash Storage, IBM Spectrum Scale or IBM Z solutions, please contact your IBM representative or IBM Business Partner, or visit the following websites: ibm.com/us-en/marketplace/scale-out-file-and-object-storage or ibm.com/z