Our mission with OpenECPDS is to keep data moving.
Inspired by operational excellence. Powered by open-source innovation.
Acquire from anywhere. Deliver everywhere. Connect with confidence. Share without limits.

OpenECPDS is a mature, production-grade data management platform originally developed at ECMWF and now open-sourced to encourage collaboration. It delivers three strategic services from a single system:

Built on container technologies, it scales from a laptop to hundreds of systems managing petabytes of data. It supports a wide range of protocols (FTP, SFTP, HTTP/S, S3, GCS, Azure Blob, MQTT/WIS2, …) and integrates with object storage, MQTT brokers, and Kubernetes.

Experience the full OpenECPDS stack in a single command. The standalone image bundles MariaDB, Master, Data Mover, and Monitor — everything pre-configured and ready to go.

Requirements: 4 GB RAM and 6 GB free disk recommended. See System Requirements for details.

docker run -d \
  --name standalone \
  -v $(pwd)/ecpds-data:/data \
  -p 7443:7443 \
  -p 7022:7022 \
  -p 8443:8443 \
  -p 8883:8883 \
  ghcr.io/ecmwf/open-ecpds/standalone:latest

The database initialises automatically on first start. The /data volume persists everything across restarts.

Note: The container uses a self-signed TLS certificate — accept the browser security warning on first visit.

Note on FTP: OpenECPDS fully supports FTP in production deployments. However, FTP passive mode (PASV) is not compatible with Docker port mapping — the server advertises its internal container address for data connections, which external clients cannot reach. For that reason, FTP is disabled in this standalone image. Use SFTP (port 7022) as a drop-in alternative for file transfers in Docker.

All examples below use the pre-configured test / test2021 account. The container uses a self-signed certificate — pass the appropriate insecure/skip-verify flag for each tool.

S3 — using AWS CLI:

aws s3 ls s3:// \
  --endpoint-url https://localhost:7443/s3 \
  --no-verify-ssl \
  --no-sign-request \
  --request-payer \
  # or with credentials:
aws configure set aws_access_key_id test
aws configure set aws_secret_access_key test2021
aws s3 ls s3:// --endpoint-url https://localhost:7443/s3 --no-verify-ssl

Or with rclone:

rclone lsd :s3: \
  --s3-provider=Other \
  --s3-endpoint=https://localhost:7443/s3 \
  --s3-access-key-id=test \
  --s3-secret-access-key=test2021 \
  --no-check-certificate

SFTP — using the sftp command-line client:

sftp -P 7022 -o StrictHostKeyChecking=no test@localhost
# Password: test2021

MQTTS — The broker uses TLS with a self-signed certificate.

Using mqttx CLI (recommended — supports true --insecure):

mqttx sub \
  -h localhost -p 8883 \
  -l mqtts \
  -u test -P test2021 \
  --insecure \
  -t '#'

Or using Mosquitto — grab the cert first via openssl (no container access needed):

# Fetch the self-signed cert directly from the broker
openssl s_client -connect localhost:8883 </dev/null 2>/dev/null \
  | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/ecpds.pem

# Subscribe (--insecure skips hostname verification)
mosquitto_sub \
  --host localhost --port 8883 \
  --username test --pw test2021 \
  --cafile /tmp/ecpds.pem \
  --insecure \
  --topic '#' -v

# All services (supervisord output)
docker logs -f standalone

# Individual service logs
docker exec standalone tail -f /data/log/master/master.log
docker exec standalone tail -f /data/log/mover/mover.log
docker exec standalone tail -f /data/log/monitor/monitor.log

Note: The standalone image runs with DEBUG logging enabled by default. This produces verbose output that is useful for exploring how OpenECPDS processes transfers, connects to data movers, and handles incoming requests. In a production deployment the log level can be controlled via the LOG_LEVEL environment variable (e.g. LOG_LEVEL=warn).

Full documentation is published at ecmwf.github.io/open-ecpds.

To build the platform from source and develop against it, you need Docker (with the default socket enabled). Tested on Linux, macOS (Intel/Apple Silicon), and Windows WSL 2.

Run make help at any time to list all available targets.

# Build the development container image, start it, and open a shell inside it
make dev

# Compile Java sources, build RPM packages, and create Docker images
make build

# Start all OpenECPDS services
cd run/bin/ecpds && make up

The monitoring UI is then available at https://localhost:3443 and the data portal at https://localhost:4443.

See Getting Started for the full walkthrough, including log inspection, stopping services, IDE setup, and Kubernetes deployment.

# Inside the development container, after 'make build':
make build-standalone

This stages the RPMs into docker/ecpds/standalone/ and builds the ecpds/standalone:<tag> image locally.

Copyright 2022–2026 ECMWF. Licensed under the Apache License 2.0.