AIMIP Phase 1: Systematic evaluations of AI weather and climate models

Brian Henn
Chris Bretherton
Nikolay Koldunov
Christian Lessig
Maria J. Molina
Troy Arcomano
Oliver Watt-Meyer
Guillaume Couairon
Renu Singh
Robert Brunstein
Yana Hasson
Antonia Jost
Noah Brenowitz
Peter Manshausen
Dale Durran
Nathaniel Cresswell-Clay
Kyle Joseph Chen Hall
Janni Yuval
Dmitrii Kochkov
Stephan Hoyer
arXiv (2026)

Abstract

We present the AI weather and and climate model intercomparison project (AIMIP), phase 1. Drawing from the rich tradition of model intercomparisons in climate model development, we specify a common experiment, output data format, and training constraints (namely, training against historical reanalysis data) for AIMIP Phase 1 models. We aim to identify differences between modeling frameworks and AI architectural choices that influence model behavior, and build trust in AI weather and climate models through open data and evaluation. AIMIP Phase 1 models must simulate the atmosphere given specified historical sea surface temperatures over 1979-2024; we then evaluate the models’ performance using five major evaluation criteria: biases, trends, response to El Niño-related sea surface temperature anomalies, temporal variability, and out of sample generalization tests. We find that AI models are able to simulate the historical climate and response to forcing as well as a conventional physically-based model, but some AI models underestimate historical warming trends, and their predictions diverge in the out of sample generalization tests. We describe the AIMIP Phase 1 dataset that is publicly available for additional
evaluations.
×