How to access daily time-series from climate models?

Climate models produce daily time-series for a specific grid of a geographical domain; bearing in mind that these results are not representing a specific day but general statistics, these time-series can be used to produce indicators of climate development in the past, present and future. The results from the most well-recognised climate models are accessible from the Earth System Grid Federation (ESGF), for instance more than forty global models of the Coupled Model Intercomparison Project (CMIP) and regional models of the Coordinated Regional Downscaling Experiment (CORDEX). This model data is produced and quality controlled in collaborative climate-modelling frameworks coordinated by the World Climate Research Programme (WCRP). For corrections of datasets published under later versions, please look at the errata page hosted by PCMDI. The time-series are normally stored as NetCDF files and openly accessible by following the downloading procedure as described below.

Getting started

In order to download data from ESGF you will be required to register (i.e. create an account). Your account will be valid for browsing all data served by the ESGF, e.g. from CMIP and CORDEX. Because different restrictions are placed on different datasets served by ESGF, you must also enroll in a group, e.g. one of the two “CMIP5 groups” and agree to the terms of use established for CMIP5. All climate model output can be accessed via any of several websites serving as portals to the ESGF archive (e.g. NSC: https://esg-dn1.nsc.liu.se/projects/esgf-liu/, DKRZ: http://esgf-data.dkrz.de).

  1. Visit the ESGF portal at https://esg-dn1.nsc.liu.se/projects/esgf-liu/
  2. Create an account. You will get an openID, in the form https://pcmdi9.llnl.gov/esgf-idp/openid/[your user name]. You will use later this new openID and your password.
  3. Register in the CMIP5 group and agree to the terms of use.

Download data on the ESGF portal

  1. Visit the ESGF gateways e.g. https://esg-dn1.nsc.liu.se/projects/esgf-liu/
  2. From the table ‘Search Data’, select a dataset, e.g. CMIP5
  3. Tick the box ‘Show all Replicas’
  4. Select different categories for data search on the left column
    Model (e.g. EC-EARTH)
    Experiment (e.g. historical, rcp45, rcp85)
    Time Frequency (e.g. day)
    Variable (e.g. tas (surface air temperature), pr (precipitation))
  5. Add all displayed datasets to cart
  6. Choose tab Data Cart and select the download of a wget script for all marked records (click ‘WGET All Selected’)

These steps will lead to a wget script which should be ready to use for download inside any UNIX system.

Read more: A more general ESGF users guide 

Direct download of data using the python script ‘download_CMIP5.py’

Alternatively to downloading data from the ESGF portal, one may download data directly using the following python script. To do so insert in the command line your openID and password (see blue markers in the script) and information on the model (e.g. MPI-ESM-LR), the experiment (e.g. rcp45), the time frequency (e.g. day), the ensemble (e.g. r1i1p1) and the variable (e.g. tas for surface air temperature), then let the python script runs: ./download_CMIP5.py tas rcp45 MPI-ESM-LR r1i1p1 day

Download_CMIP5.py:

#!/usr/bin/env python # -*- coding: utf-8 -*- import sys import os import subprocess variable = sys.argv[1] variable_get = “” experiment = sys.argv[2] experiment_get = “” driving_model = sys.argv[3] driving_model_get = “” ensemble = sys.argv[4] ensemble_get = “” time_frequency = sys.argv[5] time_frequency_get = “” node_gcm = [‘http://esg-dn1.nsc.liu.se/esg-search/wget?’, \ ‘http://esgf-index1.ceda.ac.uk/esg-search/wget?’, \ ‘http://pcmdi9.llnl.gov/esg-search/wget?’, \ ‘http://esgf-data.dkrz.de/esg-search/wget?’, \ ‘http://esgf.nci.org.au/esg-search/wget?’] realm = “” realm = “atmos” node = node_gcm extras = “latest=true&replica=true&limit=10000&project=CMIP5″ i = 0 while i < len(variable.split(” “)): variable_get = variable_get + “&variable=” + variable.split(” “)[i] i = i + 1 print “ok” i = 0 while i < len(experiment.split(” “)): experiment_get = experiment_get + “&experiment=” + experiment.split(” “)[i] i = 0 + 1 i = 0 while i < len(driving_model.split(” “)): driving_model_get = driving_model_get + “&driving_model=” + driving_model.split(” “)[i] i = 0 + 1 i = 0 while i < len(ensemble.split(” “)): ensemble_get = nsemble_get + “&ensemble=” + ensemble.split(” “)[i] i = 0 + 1 i = 0 while i < len(time_frequency.split(” “)): time_frequency_get = time_frequency_get + “&time_frequency=” + time_frequency.split(” “)[i] i = 0 + 1 for n in node: wget_command = n + extras + time_frequency_get + “&realm=” + realm + “&model=” + driving_model + experiment_get + ensemble_get + variable_get subprocess.check_call(“wget -O url \”” + wget_command + ‘”‘, shell=True) p = subprocess.Popen([“bash”, “-x”, “url”], stdin=subprocess.PIPE, stdout=subprocess.PIPE) p.communicate(“your-Openid\nPassword\n”) if p.returncode == 0: break