The REF officially launched at the CMIP conference and we got an exciting look at how this tool could transform and expedite multi-model evaluation.
The REF stands for Rapid Evaluation Framework and is a new tool created by the Model Benchmarking and Evaluation Task Team to streamline and standardize multi-model comparisons. Via a dashboard, users can select to view assessments of different model diagnostics and compare to observational datasets.
Using the REF is a great starting point for analysis, for example those deciding on research topics can get an overview of the data. The diagnostics are collected into 5 themes: ‘Ocean & Sea Ice’, ‘Land & Land Ice’, ‘Atmosphere’, ‘Earth System’ and ‘Impacts & Adaptations’. Each diagnostic also has a technical and plain language description, including why this diagnostic matters, making it accessible to new users and end users. This approach provides a centralized, user-friendly entry point for researchers working with CMIP data.
The platform’s dashboard includes a range of interactive features. Users can hover over data points to view specific model values, while a search function allows filtering by individual models and other criteria. Data can be downloaded in CSV or netCDF format, and for advanced users, a containerized local version offers flexibility to customize the API to specific needs. When publishing results derived from REF data, users are advised to cite the REF paper, its Zenodo DOI, and the individual models used.
The Read the Docs page gives a useful intro to using the REF for your own needs including the containerized version that can be adapted. We also got a Q&A to ask our immediate questions:
- If it often feels like you spend most of your time processing CMIP data rather than doing the interesting analysis, the REF may help reduce this time. For example, transformation of different model coordinates has been done for you!
- Models with incorrect metadata that cannot be added to the automated process may be rejected. Modelling centers should check everything is set up correctly.
- Legacy records will be kept (currently not operational) so when there are updates, for example if new data is added or bug fixes occur, it is still possible to see old versions. Therefore, published data can be linked back to the source used at the time, even if updates mean the current version has changed.
- The site is updated daily so as new data is processed, it will appear on the site promptly
Beyond the launch, a lot is still happening. The REF will continue to grow with help from the community, therefore suggestions of new diagnostics, new functionalities and new data additions, such as additional observational datasets, are welcomed. A committee will be established to prioritize and coordinate requests. Additional comments and suggestions can be posted on GitHub (https://github.com/Climate-REF/climate-ref).
As CMIP7 data arrives, this will of course become available on the platform. Check out the trial version with CMIP6 data here: https://dashboard.climate-ref.org/
