The Model Benchmarking Task Team met last week in a hybrid workshop to finalise the scope of a new Rapid Evaluation Tool for the CMIP AR7 Fast Track, which will support the next IPCC assessment cycle. The workshop was kindly hosted by Deutsches Zentrum für Luft- and Raumfahrt (DLR), the German Aerospace Center, in Oberpfaffenhofen, Germany. Key outcomes of this fast paced workshop included the scope for the suggested AR7 Fast Track Rapid Evaluation Framework and a collection of observational requirements for climate model benchmarking.
Dreaming Big
The Task Team members had multiple sessions to develop and refine a vision for the next generation of climate model benchmarking, where a dynamically configured set of society-relevant metrics and diagnostics are executed on demand and results are delivered to climate scientists, the climate impacts community, stakeholders, and interested members of the public.
Birgit Hassler, Task Team co-lead summarised,
“We envisage a system, designed from the outset, to be inclusive by enabling people all around the world to contribute to climate research and better understand the implications for their region. If the system also takes some burden off the modelling centres, it would be a wonderful bonus.”
Rapid Evaluation Framework
The Task Team members devised a minimum option for the Rapid Evaluation Framework that could be operational to meet the ambitious timeline for delivery of the AR7 Fast Track. This was presented during the workshop to the CMIP Panel and WGCM Infrastructure Panel receiving a positive response and useful input to the design and architecture from panel members . The Task Team members are keen for the community to go beyond the minimum and believe their ideas for an optimal framework could open up a new era of model generation and benchmarking.
Forrest Hoffman, Task Team co-lead said,
“Building this framework to leverage a variety of community benchmarking tools will be a challenge, but it will have a huge pay-off for CMIP and plans for operationalisation of CMIP.”
The framework is dependent upon community agreement on a priority set of diagnostics, considered crucial for addressing the science questions in AR7, and the Task Team hope they will be taken into consideration by the Data Request Task Team’s five thematic author teams to define a collection of harmonised variables.
The team’s plans will necessitate early coordination of the quality assurance and quality checking of the AR7 Fast Track ensemble across the data publishing modelling centres.
Birgit Hassler said,
“If we get the data quality enhanced and a first set of evaluation metrics prioritised, I think we can create something brilliant for the community to use. ”
Discussion of observation requirements
Discussion on observation requirements essential for the model benchmarking and evaluation of the AR7 Fast Track resulted in the following key recommendations for the CMIP Panel and ESMO Scientific Steering Group to take forward:
- Coordinate community uncertainty characterisation: Offering standards and coordination for characterising uncertainties in data
- Improve robustness of benchmarking: Formulate a list of datasets used by modelling centres to evaluate model fidelity during development and distinguish between forcing and benchmarking data.
- Coordinate modelling and observational communities: Regularly convene the modelling and observational communities to share new model features and new observational developments to facilitate design of new model benchmarks.
- Identify observations needed for AR7 Fast Track: Survey the science community to identify model benchmarks and corresponding observational data requirements for the AR7 Fast Track.
Members of the ESMO SSG welcomed the work of the Task Team and the recommendations during a presentation to SSG members as part of the workshop. The Task Team has undertaken to refine their list of observation requirements over the coming months.
Opportunities for engagement with the modelling centres and with the wider community are under development by the CMIP IPO. To keep up to date with the latest news and event please sign up to the CMIP mailing list.
Reflecting on the week at DLR, Forrest Hoffman said,
“The workshop was a fantastic opportunity to work more closely and efficiently with our colleagues in the task team and to make significant progress in a short time through intensive engagement with these researchers. ”
Particular thanks to the local DLR host, Birgit Hassler, and CMIP IPO facilitator, Briony Turner, for a successful workshop!
