OceanBench is a benchmarking tool to evaluate ocean forecasting systems against reference ocean analysis datasets (such as 2024 GLORYS reanalysis and GLO12 analysis) as well as observations.

OceanBench's scientific paper is published in NeurIPS 2025 and is accessible at https://openreview.net/forum?id=wZGe1Kqs8G.

@inproceedings{
  aouni2025oceanbench,
  title={OceanBench: A Benchmark for Data-Driven Global Ocean Forecasting systems},
  author={Anass El Aouni and Quentin Gaudel and Juan Emmanuel Johnson and REGNIER Charly and Julien Le Sommer and van Gennip and Ronan Fablet and Marie Drevillon and Yann DRILLET and Pierre Yves Le Traon},
  booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
  year={2025},
  url={https://openreview.net/forum?id=wZGe1Kqs8G}
}

The official score table is available on the OceanBench website.

You can train your model with GLORYS reanalysis. From an environment with OceanBench installed, run:

import oceanbench
oceanbench.datasets.reference.glorys_reanalysis()

to open GLORYS dataset as a xarray.Dataset:

<xarray.Dataset> Size: 5TB
Dimensions:    (depth: 50, latitude: 2041, longitude: 4320, time: 366)
Coordinates:
  * depth      (depth) float32 200B 0.494 1.541 2.646 ... 5.275e+03 5.728e+03
  * latitude   (latitude) float32 8kB -80.0 -79.92 -79.83 ... 89.83 89.92 90.0
  * longitude  (longitude) float32 17kB -180.0 -179.9 -179.8 ... 179.8 179.9
  * time       (time) datetime64[ns] 3kB 2024-01-01 2024-01-02 ... 2024-12-31
Data variables:
    thetao     (time, depth, latitude, longitude) float64 1TB dask.array<chunksize=(28, 1, 512, 2048), meta=np.ndarray>
    so         (time, depth, latitude, longitude) float64 1TB dask.array<chunksize=(28, 1, 512, 2048), meta=np.ndarray>
    uo         (time, depth, latitude, longitude) float64 1TB dask.array<chunksize=(28, 1, 512, 2048), meta=np.ndarray>
    vo         (time, depth, latitude, longitude) float64 1TB dask.array<chunksize=(28, 1, 512, 2048), meta=np.ndarray>
    zos        (time, latitude, longitude) float64 26GB dask.array<chunksize=(28, 512, 2048), meta=np.ndarray>
Attributes:
    source:       MERCATOR GLORYS12V1
    institution:  MERCATOR OCEAN
    comment:      CMEMS product
    title:        daily mean fields from Global Ocean Physics Analysis and Fo...
    references:   http://www.mercator-ocean.fr
    history:      2023/06/01 16:20:05 MERCATOR OCEAN Netcdf creation
    Conventions:  CF-1.4

The evaluation of a system consists of the sequential execution of a Python notebook that runs several evaluation methods against a set of forecasts (produced by the system), namely the challenger dataset, opened as an xarray Dataset.

The OceanBench documentation describes the shape a challenger dataset must have, as well as the definitions of the methods used to evaluate systems.

All official challenger notebooks are maintained and remain executable in order to update the scores with new OceanBench versions (all official challengers are re-evaluated with each new version).

To officially submit your system to OceanBench, please open an issue on this repository attaching one of the following:

1. The executed notebook resulting from an interactive or programmatic evaluation.
2. A way to access the system output data in a standard format (e.g. Zarr or NetCDF).
3. A way to execute the system code or container along with clear instructions for how to run it (e.g., input/output format, required dependencies, etc.).

In addition, please provide the following metadata:

• The organization that leads the construction or operation of the system.
• A link to the reference paper of the system.
• The system method. For example, "Physics-based", "ML-based" or "Hybrid".
• The system type. For example, "Forecast (deterministic)" or "Forecast (ensemble)".
• The system initial conditions. For example, "GLO12/IFS".
• The approximate horizontal resolution of the system. For example, "1/12°" or "1/4°".

Checkout this notebook that evaluates a sample (two forecasts) of the GLONET system on OceanBench. The resulting executed notebook is used as the evaluation report of the system, and its content is used to fulfill the OceanBench score table.

You can replace the cell that opens the challenger datasets with your code and execute the notebook.

You will need to install OceanBench manually in your environment.

pip install oceanbench

git clone git@github.com:mercator-ocean/oceanbench.git && cd oceanbench/ && pip install --editable .

You can open and manually execute the example notebook in EDITO datalab by clicking here:

Once installed, you can evaluate your system using python with the following code:

import oceanbench

oceanbench.evaluate_challenger("path/to/file/opening/the/challenger/datasets.py", "notebook_report_name.ipynb")

More details in the documentation.

Running OceanBench to evaluate systems with 1/12° resolution uses the Copernicus Marine Toolbox and therefore requires authentication with the Copernicus Marine Service.

If you're running OceanBench in a non-interactive way, please follow the Copernicus Marine Toolbox documentation to login to the Copernicus Marine Service before running the bench.

Your help to improve OceanBench is welcome. Please first read contribution instructions here.

Licensed under the EUPL-1.2 license.

Implemented by:

As part of a fruitful collaboration with:

Powered by: