Skip to content

Latest commit

 

History

41 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ISIMIP Climate Downloader logo

ISIMIP Climate Downloader

Regional downloads and structural checks for selected ISIMIP3b climate data

Python 3.10+ Data Format

A small command-line tool for downloading daily, bias-adjusted ISIMIP3b atmospheric data by model, variable, scenario, period, and geographic bounding box. Regional requests use the official ISIMIP Files API, so a multi-gigabyte global file does not need to be downloaded before cropping.

Highlights

  • Server-side regional cutouts through the official ISIMIP API
  • Atomic outputs, with resume support for large full-file downloads
  • Built-in mappings for five climate models and their supported time blocks
  • Concurrent downloads with clear failure reporting
  • NetCDF structural checks for dimensions, variables, dates, and spatial bounds
  • Optional xarray combination and climate4R NcML generation command
  • A non-interactive CLI suitable for scripts and scheduled jobs

Quick start

Python 3.10 or newer is sufficient for downloading. The wrapper examples require Bash on Linux, macOS, or Windows Subsystem for Linux. From the repository root:

bash script/isimip.sh \
  -m GFDL-ESM4 \
  -v tas \
  -s historical \
  -x "12 12.6" \
  -y "41.5 42.1" \
  -b 2011 \
  -e 2014

This requests a small regional cutout, saves it under isimip_data/, and does not prompt for a Conda environment.

Important

Omitting -x and -y downloads complete global files. A single ten-year daily file can be around 2 GB. Use a bounding box unless global data are genuinely required.

Command-line options

Usage: isimip.sh -m MODEL -v "VARIABLE [VARIABLE ...]" -s SCENARIO [OPTIONS]

Required:
  -m MODEL       GFDL-ESM4, MPI-ESM1-2-HR, IPSL-CM6A-LR, MRI-ESM2-0,
                 UKESM1-0-LL, or all
  -v VARIABLES   hurs huss pr prsn ps tas tasmax tasmin
  -s SCENARIO    historical, ssp126, ssp585, or all

Options:
  -x "WEST EAST" Longitude bounds
  -y "SOUTH NORTH" Latitude bounds
  -b YEAR         First requested year
  -e YEAR         Last requested year
  -o DIRECTORY    Output directory (default: isimip_data)
  -j WORKERS      Concurrent downloads/API jobs (default: 3)
  -c              Combine files after download
  -n ENVIRONMENT  Create NcML using this Conda environment
  -h              Show help

The year options select every published file block that overlaps the requested range. For example, -b 2011 -e 2011 downloads the published 2011–2014 block. Without year options, this project defaults to 1971–2014 for historical and 2021–2100 for future scenarios. Set -b 2015 to include the future-scenario 2015–2020 transition block.

More examples

Download two variables for a European bounding box:

bash script/isimip.sh \
  -m MPI-ESM1-2-HR \
  -v "tas pr" \
  -s ssp126 \
  -x "-10 40" \
  -y "35 70" \
  -b 2021 \
  -e 2050 \
  -o climate_data

Download and combine the selected files. Run this in a Python environment that contains xarray, netCDF4, and dask:

conda run -n isimipclim bash script/isimip.sh \
  -m GFDL-ESM4 \
  -v "tas pr" \
  -s historical \
  -x "12 13" \
  -y "41 42" \
  -b 2001 \
  -e 2014 \
  -c

Create optional NcML with an existing Conda environment containing R and the climate4R loadeR package:

bash script/isimip.sh \
  -m GFDL-ESM4 \
  -v tas \
  -s historical \
  -x "12 13" \
  -y "41 42" \
  -b 2011 \
  -e 2014 \
  -n climate4R

The -n option is explicit and optional. It uses conda run; it does not activate an environment or alter the current shell.

Validate downloaded files

Validation requires xarray and a NetCDF backend such as netCDF4:

python script/isimip.py validate \
  isimip_data/GFDL-ESM4/historical/*_cropped.nc \
  -v tas \
  --bbox 12 12.6 41.5 42.1

The validator checks that the file has a NetCDF/HDF5 signature, non-empty time, lat, and lon dimensions, chronological time coordinates, the requested variable, and coordinates inside the requested bounds. It prints a JSON summary containing variable dimensions, data types, units, standard names, and compression metadata.

This is a structural sanity check, not a complete ISIMIP protocol or scientific quality-control tool. It does not certify metadata completeness, physical value ranges, or scientific suitability.

Output layout

isimip_data/
├── GFDL-ESM4/
│   └── historical/
│       └── gfdl-esm4_..._tas_global_daily_2011_2014_cropped.nc
├── combined/
│   └── historical/
│       └── GFDL-ESM4_combined.nc
└── ncml/
    └── historical/
        └── GFDL-ESM4_historical.ncml

Regional files receive the _cropped.nc suffix. Full global downloads retain the original ISIMIP filename.

Python API

The downloader can also be imported directly:

from script.isimip import download_isimip_data

files = download_isimip_data(
    model_choices="GFDL-ESM4",
    variables=["tas", "pr"],
    scenario="historical",
    bbox=(12.0, 12.6, 41.5, 42.1),
    start_year=2011,
    end_year=2014,
    output_dir="isimip_data",
)

Optional environments

Downloading and server-side subsetting use only the Python standard library. Additional features need:

Feature Packages
Validate NetCDF xarray, netCDF4
Combine files xarray, netCDF4, dask
Generate NcML Conda, R, climate4R loadeR

For example, create a Python analysis environment with Mamba:

mamba create -n isimipclim -c conda-forge python=3.12 xarray netcdf4 dask

See the climate4R documentation for its current R installation instructions.

Test the project

The standard-library regression suite needs no extra packages:

python -m unittest discover -s tests -v

To test a real download, use the small Quick start bounding box and then run the validator in an environment containing xarray and netCDF4.

Verification scope

The current implementation has been checked with:

  • Unit tests for file-block selection, model mapping, invalid bounds, and download failure propagation
  • A live GFDL-ESM4 regional download for tas and pr
  • Inspection and combination of the downloaded NetCDF files
  • URL availability checks for all five supported models

Complete global downloads were not run because individual files can approach 2 GB. NcML generation was not run because the available Conda environments did not contain climate4R loadeR; that path requires a user-provided environment. The supported models, variables, and time blocks are defined in the source and may need updating if the upstream repository changes.

Data source and citation

The file naming and directory structure follow the ISIMIP3 protocol and data are retrieved from the ISIMIP Repository. Consult the repository entry for the dataset version, license, and citation that apply to data used in research.

This project is a convenience client and is not an official ISIMIP product.

About

bash script to download ISIMIP3 climate data and aggregate into NcML files

Topics

Resources

Stars

1 star

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages