CMIP Rapid Evaluation Framework

Version 1 of the Rapid Evaluation Framework (REF) containing CMIP6 data was launched at ESM2025 on the 07 October 2025. The CMIP7 ready version was launched at the CMIP 2026 Community Workshop on the 09 March 2026, press release here. You can access it through the dedicated dashboard. A tour of the dashboard is available here. The containerised version installation is available here.  For tips on using the REF, take a look at: Best Practice for Using the CMIP Rapid Evaluation Framework Output.

What is the CMIP7 Assessment Fast Track Rapid Evaluation Framework?

As climate models become increasingly complex, there is a need for systematic and comprehensive evaluation and benchmarking through comparison with best-available observational data to assess model fidelity across a wide range of variables.

Vision for the Rapid Evaluation Framework: A community owned evaluation framework, built upon, and compatible with, existing community evaluation packages that incorporates an application programming interface (API) for executing metrics generated from community evaluation packages, across the globe. 

Goal: The CMIP7 Assessment Fast Track Rapid Evaluation Framework (CMIP7 AFT REF) will be a complete end to end system providing a systematic and rapid performance assessment of the expected models participating in the CMIP7 Assessment Fast Track, supporting the seventh IPCC Assessment Report (AR7) cycle.

Outcome: The REF provides the ability to fully integrate evaluation tools into the CMIP publication workflow, and their diagnostic outputs published alongside the model output on the Earth System Grid Federation (ESGF) through an easily accessible website. 

Wider community use beyond CMIP: The REF is designed to be a starting point for the community to develop and build upon, with applications across the WCRP Modelling Multiverse and beyond.

Notice regarding minimum data threshold for CMIP7

The REF should not be used to draw conclusions about the CMIP7 ensemble until a robust threshold of minimum data has been reached. This number is determined from the number of modelling centres submitting simulations and simulations from individual models. Both numbers (individual modelling centres AND individual models) need to be at least 3 (with at least 1 ensemble member each).

If there are results displayed in the REF based on these thresholds there will be a warning provided.

The warning will be removed if both thresholds exceed 20 (so 20 individual modelling centres AND 20 individual models).

Message to display until threshold reached:

There is only limited data available at the moment and the results are therefore considered preliminary.

This notice is in accordance with Decision ID 101 approval by CMIP Core Panel, available in the CMIP decision log.

Original vision for the technical architecture of the REF

When the REF was initially scoped out, elements that were deemed essential were identified, alongside elements that would be nice-to-have (desirable) if additional community resources could be found to develop them. These are distinguished in the the figure below.

Who is funding, delivering and governing the REF?

This project has been made possible by resources from European Space Agency and US Department of Energy and the expertise and time in kind of the following software packages and organisations: Climate Resource, eScience Center, ESMValTool, ILAMB, IOMB, PMP , CMEC, DLR, National Centre for Earth Observation (NCEO), Science and Technology Facilities Research Council RAL Space and CEDA

A new scientific panel is planned to be set up in 2026 by the WCRP Earth System Modelling and Observations (ESMO) core project to provide governance and scientific oversight. This panel will liaise closely with the CMIP Panel for all matters relating to CMIP.

What diagnostics will be included?

For the Assessment Fast Track REF, the CMIP Model Benchmarking Task Team, through consultation with the CMIP modelling community and representatives from the observational community, have collected a set of informative diagnostics across five thematic areas (view the selected diagnostics here). These align with the thematic groupings of the CMIP7 Harmonised Data Request – atmosphere, Earth system, impacts & adaptation, land & land ice, and ocean & sea ice.

Accessing the REF results

The Rapid Evaluation Framework (REF) provides diagnostics to characterise climate model performance and highlight model spread, diversity and differences. These results may help researchers identify models suitable for specific applications but should not be interpreted as identifying “good” or “bad” models. The REF serves as a starting point for deeper exploration and investigation into CMIP6 and CMIP6Plus model output as well as that of CMIP7 in due course, once it is published to ESGF.

You can access the REF results through the REF Dashboard. A tour of the dashboard is available here. The containerised version installation is available here: https://climate-ref.readthedocs.io/

The Climate REF Dashboard provides multiple interfaces for accessing evaluation results:

• Data Explorer Interactive interface for querying and visualizing model evaluation results. Users can select specific models, variables, and experiments for detailed analysis.
• Diagnostic Browser Pre-computed evaluation metrics organized by model component and diagnostic type. Results include statistical summaries, spatial maps, and time series comparisons.

Comprehensive documentation is available, including:

• Technical specifications
• Diagnostic descriptions and equations
• API documentation for programmatic access
• Example workflows and use cases

How can I contribute?

There are only four software packages contributing to the CMIP7 AFT REF. However, the technical delivery team are building the framework in a modular manner that will allow for additional software packages at a later date. Please complete our Expression of Interest form if you would like to become involved as a:

• Tier 1 Node – willing to publish and replicate the REF.
• Modelling centre – participating in the development work either through our stress testing exercise or running a containerised version.
• Reference Data Provider – providing reference data sets.
• Scientist – providing a “sanity check” and realm specific expertise.

Or have:

• Ideas for community expansion

What is the timeline of the CMIP7 Assessment Fast Track REF?

The latest overview timeline can be found here: Rapid Evaluation Framework development timeline

For further information

Please sign up for the latest news and updates on the REF through this form, selecting “Model Benchmarking mailing list”. If you have further questions please contact .

Frequently Asked Questions

Please check our community Q&A Forum which also should appear below. If you still cannot see the answer to your question, we encourage you to submit to the forum using the form below. The CMIP IPO will ensure that your question is answered by the relevant person.