Traditional forecasting still beats AI for the most extreme weather

AI is being touted as the future of weather forecasting—faster and more precise. But new research shows a major blind spot: it often fails at predicting extreme weather. Traditional physics-based models still do better. “They do perform well on a lot of tasks, but for very extreme events—that are the most important for society—they still struggle,” says Sebastian Engelke, a statistics professor at the University of Geneva and one of the authors of a new study in Science that pitted some of the leading AI weather models, including GraphCast and Pangu-Weather, against a database of recent extreme events. For record-breaking heat, like a heat wave in Siberia in early 2020 that led to wildfires and melting permafrost, AI predictions tend to underestimate high temperatures. (The heat wave would have been almost impossible without climate change; another study found that global warming made it 600 times more likely to occur.) They’re also less accurate than older models at predicting extreme wind or record-breaking cold. That’s because they’re trained using decades of past data. “They try to empirically understand, if I see a certain type of weather today, what is the weather tomorrow?” says Engelke. “Essentially, they are reproducing what has happened in the past. If we’re looking at extreme weather, and especially record-breaking events, then this has not been observed in the past. It’s really the lack of information in their training data that makes it almost impossible for them to forecast it.” The study looked at models a year ago, so they’ve already improved; some have added probabilistic models that predict multiple outcomes to try to become more accurate. But the fundamental problem still exists, because they’re still based on training data from the past. Traditional physics-based forecasting uses complex mathematical models to represent the physical world instead, and can more readily adapt to new conditions. (Traditional models aren’t perfect at pred