The Met Office has a mandate to deliver timely weather information to millions of customers across the UK. How could it ensure 24x7 availability for the systems that share weather data with customers?
The Met Office migrated its meteorological databases from x86 systems to a resilient, high-performance and scalable IBM LinuxONE platform—ensuring it can handle massive peaks in requests.
Enablesa single lean team to support a large number of core Linux applications
Cutsoperational costs through ongoing database consolidation
Ensuresmillions of customers can access critical weather data 24x7
Business challenge story
Early warnings reduce risk“The weather is critical to people’s businesses and lives—and timely warnings about adverse conditions can have a massive impact on the national economy and infrastructure,” says Richard Lawrence, Enterprise IT Architect at the Met Office. “As the national weather service provider for the UK, our mandate is to deliver timely weather warnings 24 hours a day, 365 days a year.”
Graham Mallin, Executive Head of Technology at the Met Office, continues: “Emergency services need to know what's likely to happen, when, where and how severe it's going to be. Even minutes of extra time can make a big difference to their ability to prepare and respond to a hazard, which in turn makes big differences to people’s lives and the natural environment. And since 2011, the Met Office has saved the UK economy more than GBP260 million a year.
“We have millions of customers—including the general public and more than 1,000 professional bodies responsible for aviation and emergency planning. To keep our customers informed, our meteorologists study and analyze the weather in real time, and our information systems are a key enabler of the services we provide.
“In our high-performance computing [HPC] environment, we use a large supercomputer to power our scientific analysis. At the tier below, high-end Linux servers and mainframe systems help us to convert this data into products and services for our customers. Finally, we use web and mobile applications to deliver the information to the people who need it.”
In the past, the Met Office relied on x86-processor based systems to deliver weather warnings to its customers—but increasing demand for weather information threatened to reduce the manageability, reliability and cost-effectiveness of the platform.
“A pleasant and sunny winter’s day in southeast England can quickly turn into a significant snow event—which transforms a slow trickle of users into a torrent of 15 million data requests in the space of a single hour,” says Richard Lawrence. “To help us continue to meet our mandate to deliver weather warnings as quickly as possible, we looked for a new server platform to support our data delivery systems.”
Selecting a dependable platformTo achieve its goal, the Met Office selected two IBM LinuxONE servers, each with 22 cores running Red Hat Enterprise Linux operating system. The organization worked together with IBM Premier Business Partner Computacenter Limited to deploy the new platform.
Richard Lawrence explains: “We see that LinuxONE delivers the resilience and reliability we need to build a future-ready platform to support our data delivery processes to millions of customers across the UK.”
The Met Office is now migrating key database systems—including Oracle, PostgreSQL and MongoDB databases, as well as Apache Tomcat web services and Apache ActiveMQ middleware technologies—from x86 systems to LinuxONE.
“One of the recent applications we migrated to LinuxONE is our meteorological database,” continues Richard Lawrence. “We collect about 200 million unique observations every single day from the UK and around the globe, and they are collected, processed, stored, and then reissued by a home-grown application.
“To send this information to the right part of our organization or externally around the world, we rely on a message switching system, which was previously running in a very complex x86 environment. Our weather database is already 50 petabytes, and growing by up to 100 terabytes per day. We estimate that our archive will reach a size of 500 petabytes by 2020—so the scalability of our platform is always at the front of mind. If we had continued to run the application on the x86 platform, we would have needed to scale horizontally to meet demand at peak times. However, scaling out would have caused load contention, forcing us to invest in new software to keep data moving smoothly.
“Our message switching system is right at the core of what we do, and if we can’t send the correct data to the correct places, everything falls apart. By migrating the system to LinuxONE, we gain the reliability and performance we need, without any increase in cost and complexity at the back end. Better still, we were able to complete the migration process—which we predicted would take months—within just a couple of weeks, enabling us to start experiencing the efficiency benefits rapidly.”
Delivering warnings fast—whatever the weatherWith IBM LinuxONE servers at the heart of its last-mile data delivery infrastructure, the Met Office is achieving its goal of delivering information to customers whenever they need it.
“Our web services are becoming increasingly important in delivering data to our clients, and today we have the assurance that those databases are running on the resilient LinuxONE platform,” says Richard Lawrence. “Even when we experience extreme load, LinuxONE keeps running like clockwork—it gives us a consistent set of performance that we know we can rely on. As a result, we’re confident that we can continue serving up the vital information that our customers need.”
By consolidating its data systems to the IBM platform, the Met Office is driving substantial cost savings.
“We see that over the next ten years, developers are going to have to start thinking seriously about how to make their core applications run as efficiently as possible,” comments Graham Mallin. “The database consolidation we’ve done on LinuxONE has been a huge success for us, and it’s generating some big paybacks. In fact, we have already achieved software license savings that have practically paid for the new platform.
“Crucially, we anticipate that the HPC capabilities we are developing will ultimately deliver a substantial return on investment for the UK economy. By enabling projects that improve our sustainability and adaptability to a changing climate, we estimate a return for the UK in the order of GBP2 billion over the next five years—and our LinuxONE platform will make a significant contribution to that figure.”
Richard Lawrence adds: “Currently, we are supporting our LinuxONE environment with only 2.5 full-time equivalents, which is relatively small considering the number of applications on the platform. Looking ahead, our LinuxONE environment will become the platform of choice for many of our core applications. The more we consolidate onto the LinuxONE platform, the further we will reduce our operational costs. And most importantly, we gain the resilience, robustness and guaranteed performance that comes with an enterprise-class Linux environment.”
The Met Office is now using LinuxONE as a cost-effective platform to launch new, sales-driving products.
“We’re working on a new project—named ‘decoupler’—to enable our people to create innovative, data-driven products faster,” says Richard Lawrence. “To help achieve this goal, we are building a set of APIs that give us greater governance and control over our meteorological data and systems, powered by a range of open-source technologies, including THREDDS Data Server, Jenkins, Kubernetes, Puppet, Chef and Juju. With LinuxONE, we are well positioned to deliver on our open-source strategy—particularly as support for an increasing number of open source tools becomes available on the platform. And because LinuxONE is so scalable, it will be straightforward for us to meet our considerable storage requirements for THREDDS, which we estimate will grow to 60 terabytes within just two years.”
Graham Mallin concludes: “LinuxONE plays a key role in our information systems landscape, and we are already planning ways to obtain even greater value from the platform—for example, investigating the possibility of deploying Docker for our developer community. We can bet the business on LinuxONE—and I can sleep easily in the knowledge that we can absolutely rely on our data delivery systems.”
About Met Office
Headquartered in Exeter, England and founded in 1854, the Met Office is a world-leading provider of weather and climate services. Part of the Department for Business, Energy & Industrial Strategy, the Met Office supports a wide range of industries, including energy generation, water distribution and transport, with a major focus on the aviation, insurance, retail, banking and environmental sectors. The Met Office uses over 200 million daily weather observations, an advanced atmospheric model and a high-performance supercomputer to create millions of tailored forecasts and briefings every day.
- LinuxONE - Emperor