Could AI-fueled weather forecasts boost renewable energy production?

When a heatwave hit the U.K. this summer, the National Grid’s electricity system operator was forced to fire up a coal plant, ending a 46-day streak in which fossil fuels hadn't been needed.

A nuclear power station was out of action thanks to maintenance work, wind levels were low, and — as Britons gasped in the heat and turned air conditioners on — there just wasn't enough solar power to meet surging demand.

What’s more: National Grid had a second coal plant warmed up and ready to deliver electricity, thanks to uncertainty over the short-term weather. (And this wasn’t cheap; keeping power stations on standby can run into hundreds of thousands of dollars per day, paid for ultimately by a utility’s own customers.)

The incident highlights the problems of managing an energy supply that's increasingly based on renewables, and thus increasingly hard to predict. According to the International Energy Agency, utilities still rely on the average legacy thermal power plant for 4,000 hours per year to manage seasonal variability.

While traditional weather forecasting works for a traditional grid, the wide-scale adoption of renewables — ideally without expensive fossil fuel plants as backup — calls for far more accuracy. And that's where artificial intelligence has begun to enter the picture, training a system using existing weather data so that it recognizes patterns and can use current weather conditions to make predictions.

Traditionally, weather forecasts are based on numerical weather prediction, a mathematical model based on complex equations that use physics to calculate the flows of heat, radiation and water through the atmosphere. 

Such systems suffer from a lack of fine scale, with a resolution of just over a square half-mile (one kilometer). Run via supercomputer, they demand a huge amount of energy and computing power, which makes them highly expensive to use. And perhaps most importantly, they produce predictions slowly, are sluggish to run, and are unreliable for short-term weather forecasting. 

However, according to Jack Kelly, co-founder of nonprofit Open Climate Fix, in recent years machine learning has produced alternatives to the status quo.  

"Some of these projects show impressive improvements over conventional forecasts at short forecast horizons — up to about 24 hours ahead,” Kelly said. “Once trained, ML models are very quick to make predictions, typically less than a minute."

In one such project, Open Climate Fix is currently working with National Grid’s system operator to improve the accuracy of its solar forecasts by predicting where sunlight will fall. 

This involves training a machine learning model to understand how and where clouds are moving in relation to solar arrays on the ground below, using a number of different data inputs including satellite data, traditional predictive models, national weather radar systems, and real-time solar photovoltaic data. While OCF’s focus is on the nation-wide applications of its technologies, it is also conducting trials with individual solar farms.

These enhanced models can improve predictions of supply and demand, with a fall in temperature of 1 degree Celsius tending to lead to an increase in demand of around 500 megawatts; meanwhile, heavy rain ups it by one gigawatt. They could also allow individual solar farms to improve scheduling and operational planning.

While it’s still early days, certain utilities are starting to gradually introduce AI into their existing forecasting systems. However, caveats abound. 

For instance, AI models that rely on historical data often struggle to predict extreme weather events that can wreak havoc on the grid. And training the models — especially as weather patterns grow ever more difficult to predict — can be labor-intensive.

So developers’ focus today is to merge the best of both approaches: combine numerical weather predictions of the past with the large machine learning models of the present. Kelly said Open Climate Fix’s model is already almost three times more accurate than the National Grid operator’s existing two-hour-ahead PV forecasts. The team is hoping to improve that figure by another 20 percent, while adding new features.

And Open Climate Fix is not alone in pursuing a hybrid model. Benjamin Totel, an energy trading meteorologist and hydrologist with French utility firm Engie, said forecasting based solely on AI “remains a research project so far” for the utility.  

"However, we are using weather data within AI models to improve our portfolio management on a regular basis,” he said, adding that the new technology “can bring added value."

According to Amy McGovern, director of the National Science Foundation’s AI Institute for Research on Trustworthy AI in Weather, Climate, and Coastal Oceanography, these advancements in traditional predictions can happen at multiple steps in the process.

"First, AI can be incorporated into the [numerical weather prediction] models directly in a hybrid approach, where the AI is trained and then used to replace some of the physics inside the… models,” she said, explaining that this AI-enhancement can speed them up.

But, McGovern added, AI is also being applied to the models post-processing, to adjust for biases in space and time. 

Even as the industry begins to get comfortable with the hybrid models, however, new and improved techniques are emerging, with so-called data-driven models leveraging vast amounts of weather data to make their predictions. 

Huawei's recently launched Pangu-Weather, for instance, trains an AI using three-dimensional — rather than two-dimensional — meteorological data for forecast intervals of between one and 24 hours.

Huawei claims this model is an extraordinary 10,000 times faster than traditional methods, and can accurately predict fine-grained meteorological features including humidity, wind speed, temperature, and sea level pressure in just seconds.

Pangu-Weather has already been used successfully in tests by the European Centre for Medium-Range Weather Forecasts to predict the path of Typhoon Doksuri, which made landfall in southern China in July.

But the most crucial testbed for AI’s forecasting value for utilities won’t necessarily be these every-so-often instances of extreme weather. Rather, it will be its role in day-to-day operations as variable renewables become the backbone of the grid. As that transition unfolds, AI could reduce utilities’ reliance on back-up fossil fuels and maximize the value of zero-emissions wind and solar.

"Accurate forecasts for weather-dependent generation like solar and wind are vital for us in operating a low-carbon electricity system," says Anna Carolina Tortora, head of innovation strategy and digital transformation at National Grid’s system operator.

"The more confidence we have in our forecasts, the less we’ll have to cover for uncertainty by keeping traditional, more controllable fossil fuel plants ticking over."