Which quality controls are done when producing the indicators? (CMIP6)

There is a detailed workflow to calculate the Climate Indicators (CI) provided in the Climate Information Portal. Many quality controls are completed throughout the production workflow to ensure that the indicators are of high quality.

The chart in the figure below describes the different steps in the workflow and highlights (in orange) the quality control procedures. Each procedure is adapted to the dataset it is applied to (to account for different variables, ranges and more), and can be repeated throughout the workflow. Essential Climate Variables (ECV) from Global Climate Models (GCM) are downloaded from the Earth System Grid Federation (ESGF), the largest archive of climate data world-wide. ESGF has already some standards which the climate community follows in order for the output of the climate models to be available to the scientific community.

Types of quality control procedures

The following types of quality control procedures are included in the orange boxes in the above figure:

File format checks/pre- and post-processing

The file format checking procedure is performed after pre-processing and postprocessing (steps 1,3,4 in figure above). It ensures that the files are in the right format. The following checks are made and corrected if necessary:

Data gaps/overlapping periods or missing values. The check ensures that the data is without gaps and does not have overlapping periods or missing values. Cases when a part of the time series was missing (e.g. a missing 5-year time-slice file) led to exclusion of the projection from the ensemble.
Units appropriate for each indicator; e.g. the climate model temperature unit is in general Kelvin (K), and values are then converted to Celsius (ºC).
Data dimensions: Check that all data comes on the same grid and has the same number of time steps.
Time calendar definition: Check that all data has been converted to the same calendar.
The metadata relevant to the CI production is checked to be complete, correct and follow the metadata standards. All files are edited to make metadata homogeneous.

Data pre-processing

Data pre-processing quality procedures are completed in step 1 (1 in the figure). This step is needed to make sure all datasets are in a format that can be handled within the production of the indicators. All files must be in the same format to be comparable. The following actions are the main steps in this quality control procedure:

Converting calendar to standard time reference
For bias-adjusted data: remapping data to HydroGFD3.2 grid with 0.25 degrees spatial resolution
For non-adjusted data: remapping data to a grid with 2.0 degrees spatial resolution

Range check on Essential Climate Variable (ECV) data

All ECV climate scenario data (time series of both raw and bias adjusted data) are tested against a variable-specific range in step 1,3 and 4 (see figure above). It ensures that the data do not have unrealistic values. Scenarios with data outside the range are flagged for further investigation. If a dataset has too many values outside the expected range, it can be excluded from the workflow based on expert judgement.

Evaluation of bias adjustment

Statistics for climatological periods such as mean, median, minimum, maximum values over the full ensemble are calculated for every grid point (step 2 in figure above). Experts inspect the bias adjustment performance, check for outliers in the data, and climate change signal modification. If a projection shows large deviations from non-adjusted data and/or the rest of the ensemble, it is inspected in more detail. In severe cases, this could lead to an exclusion of the projection from the ensemble.

Evaluation of climate indicators

Quality checks for climate impact indicators, both non-adjusted and bias-adjusted CI are done in steps 5 and 6 in figure.

Metadata and filenames are checked to follow the pre-defined CI-specifications.
Unreliable data is masked out, e.g. in regions where relative changes of precipitation become unreliably large due to very low values in the reference period.
CI data is evaluated against CIs derived from reference data and compared to all other projections. If any projection shows large deviations, the data is inspected in more detail by experts.