Building a power system that’s fit for the future

Central London hums with business, tourism and travel. Observe the hustle and bustle from the ornate Holborn Viaduct road bridge and you’ll feel the electrifying energy that runs through the heart of the UK’s capital city.

That feeling is no coincidence.

One of the first sparks of electricity in the UK lit the lamps adorning this very bridge over a century ago.

Today, millions of streetlights illuminate the country, all thanks to the National Energy System Operator (NESO).

NESO delivers electrical power to households, businesses and public services across Great Britain. The grid is fundamental to modern life, so the specialized team that ensures its continuous operation is undergoing a system transformation to better manage the clean energy needed to meet the UK’s net-zero greenhouse gas (GHG) emissions targets.

NESO manages the grid using a balancing system. This system ensures that electricity consumption and production are always evenly matched, the frequency is constant and sufficient reserve capacity is ready. Every second counts during the balancing process. If the grid isn’t perfectly balanced, electricity may not get to those who need it in the most efficient way.

The power system in Great Britain has become more volatile over the past decade, partly due to an increase in renewable energy sources. The country has plans to reduce emissions from power generation and support the growth of electrification of energy, whether for transport, heat or industrial processes. Renewable generation sources, such as wind and solar farms, are better for the environment but less predictable, generate smaller amounts of energy and lack inertia compared to traditional fossil fuel power generation plants.

“In 2010, there was approximately 1 gigawatt [GW] of wind generation on the grid and less than 100 megawatts [MW] of solar generation,” says Brendan Lyons, Balancing Program Director at NESO. “Over the last few years, we’ve seen a massive increase in renewable energy. The system now has over 20 GW of wind generation and over 10 GW of solar generation.”

NESO’s balancing system was originally built in the 1990s when fossil fuels were the dominant source for electricity generation. It wasn’t equipped to dispatch a large number of smaller-generation units.

To manage the grid in a green energy world, NESO has established a program of work to transform the existing balancing system with a new Open Balancing Platform (OBP), which can cater to thousands of smaller units. “We need to transform to meet four main objectives,” Lyons says. “One, to have the ability to manage a large increase in market participants. Two, to be able to adapt more quickly to new requirements, innovations and services. Three, to create a level playing field for all providers across storage, small, aggregated units and the demand side. And four, to reduce costs for consumers through the optimization of balancing costs.”

NESO began searching for a global systems integrator partner to provide assistance. Not long after, the operator had a meeting with the IBM Consulting^® team.

“We were attending an innovation day in Bangalore, India,” recalls Colin Walkinshaw, a Chief Technology Officer at IBM. “We discussed the challenges NESO was facing in the control room and co-created a business and technology vision for the future.”

This brainstorming session kickstarted a shared transformation journey between the two organizations. NESO and IBM Consulting joined forces to develop a transformation roadmap. The “rainbow team,” as Lyons refers to the collaboration, worked in lockstep to design the perfect vision for the future of energy balancing in Great Britain. What began as a plan to improve the grid’s green energy capabilities soon blossomed into something much greater.

“From day one, our objective has been to drive the transformation to a reliable, affordable, fully decarbonized electricity system that’s fair for all by 2035,” says Andrew Fletcher, an Executive Architect at IBM, and the Chief Architect for the OBP product at NESO.

The path to zero-carbon operations began with blueprinting the OBP, the secure-by-design core architectural system that would replace NESO’s existing balancing mechanism. “We identified and prioritized the core architecture; this enabled the structural integrity of OBP for incremental transformation,” says Walkinshaw.

During this phase, the platform options were stress tested with the Red Hat^® OpenShift^® (link resides outside of ibm.com) Platform, an industry leading hybrid cloud platform, because it enabled pace in delivery through automating processes and a modular microservices architecture.

The Red Hat OpenShift hybrid cloud foundation infuses all the benefits of a highly available cloud-native environment into the OBP. The platform provides the flexibility and scalability of a public cloud, such as Microsoft Azure, combined with the resiliency and security of a critical national infrastructure (CNI) in a privately held data center.

“The platform is more than a set of technologies. We have coupled it with standards and agile ways of working that have enabled development and operations to incrementally deliver business value faster with enhanced security and resilience,” says Fletcher.

Digital delivery is at the heart of NESO’s software lifecycle. Since the initial launch in December 2023, NESO’s leadership team has used their strong digital capabilities to position delivery using a more product-centric approach. Head of Delivery, Mayank Jha, and Senior Delivery Manager, Nisha Bhamidimarri, have helped drive relentless, continuous improvements in the OBP’s delivery, moving closer to a true product delivery model through a shared vision while delivering business value each sprint.

Bhamidimarri says, “we have adopted a more detailed and data-driven process for priorities in our backlog to meet the shifting demands of the industry with value at the heart of all our decisions. Since launch, we have achieved so much and are now able to realize our true vision. Our digital credentials have also enabled us to showcase our delivery capability, which others within the organization have benefited from.”

NESO’s combined way of working has enabled the execution of its product vision at pace, developing and testing applications using the Red Hat OpenShift modular architecture on the hybrid cloud. This process is done without impacting the grid while keeping the control room, the regulator and the industry up to date through compendious agile reporting.

Jha adds, “having the ability to respond to market changes faster and with no outages is a game-changer for NESO. This is all directly relatable to our digital ways of working, coupled with the platform we have implemented. Additionally, these efficiencies will enable us to harmonize services, which was impossible before, and will add real value to consumers.”

The use of public cloud services provides the flexibility of rapid development and testing that’s not possible with a physical data center. Jack Oliver, an IBM Associate Partner and Complex Delivery Lead for the balancing transformation explains, “this delivery approach has enabled NESO to deliver functional change to the control room every sprint, with no downtime associated with releases, representing a fundamental shift in our mindset. This year alone, we have delivered 18 sprint releases, across more than 200 new microservice versions,” he concludes.

Since building the OBP, NESO has been able to make better use of its current energy sources while designing a roadmap that caters to a more diverse set of energy sources in the future. The organization has also managed an increase in the number of generation units in its network and fundamentally changed the way the grid is balanced.

Using OBP’s automation tools, NESO has enabled the bulk dispatching of instructions, a capability not present in its previous system. Using this process, the operator can send hundreds of dispatch instructions to smaller-generation units simultaneously.

“When comparing the 3 months before OBP went live in December 2023 to the last 3 months—June, July and August of 2024—the average dispatch volume [MWh per day] of batteries increased from 659 to 2,527. Daily instructions have increased from 217 to 1,867. This represents a 54 GWh increase in dispatch volume for BESS [battery energy storage system] in the balancing mechanism for August 2024 alone compared to a 251 GWh increase in total volume since launch,” says Bernie Dolan, Principal Product Manager for the Open Balancing Platform.

He continues, “assuming electricity generation creates 149 gCO2/kWh in GB[1] and an increase in battery dispatch saves the equivalent CO2, the introduction of the OBP has saved an estimated 8,046 metric tons of CO2 in August 2024.” And since its December launch, an estimated 37,400 metric tons of CO2 has been saved.

In addition to the sustainability metrics Dolan mentioned, Lyons also notes, “this new level of balancing optimization has led to significant savings for NESO and electricity consumers in Great Britain. Consumers are expected to save GBP 15 million per year in energy spending thanks to the enhanced system.”

The OBP gives NESO the control to reserve excess green energy using battery storage sites. Surpluses in wind or solar energy can be dispatched to batteries, which are then charged for later use. The implementation of battery units reduces reliance on gas-fired generation in times of need by ensuring that renewable energy doesn’t go to waste.

The OBP’s hybrid cloud architecture centralizes the complex functions and capabilities that make up the system, which provides a cohesive experience across the control room. Thanks to the newly designed digital operator interface, engineers at NESO can now choose from a list of preselected units to meet network requirements. This simplified process increases dispatching speed and reduces the need for manual instructing. “We observed during testing that what would have taken over 30 minutes and 610 clicks to instruct in the legacy system took just 3 minutes and 10 clicks in OBP,” says Lyons.

The maturity the team built in delivery and the radical focus on continuous improvements enabled them to launch OBP as planned. Moreover, through the automation and agile principles at the heart of the delivery, NESO is enabling changes to the control room with every sprint—a fundamental delivery shift. These results were further recognized at the 2024 Computing.co.uk DevOps Excellence Awards, where the OBP team won the award for Best DevOps Team of the Year.

Though much has already been accomplished, this phase is only the first in NESO’s sustainability journey. The OBP will continue optimizing the operation of the UK’s power system, driving the journey toward zero-carbon operability by 2050.

NESO (link resides outside of ibm.com) is the electricity system operator for Great Britain. The organization is responsible for delivering electricity to households and businesses throughout the country. Its complex energy system must be planned and operated in a way that considers the interactions across electricity, gas and other forms of energy.

Fundamental to NESO is the ability to bring an independent, impartial voice to energy system planning and operations that takes a whole system view.

The organization brings together eight activities required to deliver the plans, markets and operations of the energy system of today and the future. Bringing these activities together in one organization encourages holistic thinking on the most cost-efficient and sustainable solutions customers need.