On this Page
What is the CMIP AR7 Fast Track Rapid Evaluation Framework?
As climate models become increasingly complex, there is a need for systematic and comprehensive evaluation and benchmarking through comparison with best-available observational data to assess model fidelity across a wide range of variables.
Vision for the Rapid Evaluation Framework: A community owned evaluation framework, built upon, and compatible with, existing community evaluation packages that incorporates an application programming interface (API) for executing metrics generated from community evaluation packages, across the globe.
Goal: The CMIP AR7 Fast Track Rapid Evaluation Framework (AR7 FT REF) will be a complete end to end system providing a systematic and rapid performance assessment of the expected models participating in the CMIP AR7 Fast Track, supporting the next IPCC Assessment Report 7 (AR7) cycle.
Outcome: The REF provides the ability to fully integrate evaluation tools into the CMIP publication workflow, and their diagnostic outputs published alongside the model output on the Earth System Grid Federation (ESGF) through an easily accessible website.
Wider community use beyond CMIP: The REF is designed to be a starting point for the community to develop and build upon, with applications across the WCRP Modelling Multiverse and beyond.
Who is funding, delivering and governing the REF?
This project has been made possible by resources from European Space Agency and US Department of Energy and the expertise of the following software packages and organisations: Climate Resource, eScience Center, ESMValTool, ILAMB, IOMB, PMP and CMEC.
An interim scientific steering panel will be established early in 2025 to provide governance and scientific oversight. This panel will liaise closely with the WCRP Earth System Modelling and Observations (ESMO) core project.
What diagnostics will be included?
The CMIP Model Benchmarking Task Team, through consultation with the CMIP modelling community and representatives from the observational community, have collected a set of informative diagnostics across five thematic areas (view the selected diagnostics here). These align with the thematic groupings of the CMIP7 Harmonised Data Request – atmosphere, Earth system, impacts & adaptation, land & land ice, and ocean & sea ice.
How can I contribute?
There are only four software packages contributing to the AR7 FT REF. However, the technical delivery team are building the framework in a modular manner that will allow for additional software packages at a later date. Please complete our Expression of Interest form if you would like to become involved as a:
- Tier 1 Node – willing to publish and replicate the REF.
- Modelling centre – participating in the development work either through our stress testing exercise or running a containerised version.
- Reference Data Provider – providing reference data sets.
- Scientist – providing a “sanity check” and realm specific expertise.
Or have:
- Ideas for community expansion
What is the timeline of the AR7 Fast Track REF?
For further information
Please sign up for the latest news and updates on the REF through this form, selecting “Model Benchmarking mailing list”. If you have further questions please contact .
