A tool for testing search relevance.

Python 3.12 (pyenv recommended) is a prerequisite.

You can install with pip (or pipx) just by doing

pip install git+https://github.com/wellcomecollection/rank.git

Install uv, then run uv sync in the root of the repo. This creates a local .venv/ with the project and development dependencies.

Either activate the virtual environment (source .venv/bin/activate) or prefix commands with uv run.

Note that the tool uses the Wellcome Collection Catalogue _elasticConfig endpoint to determine which index to query against. The --query parameter should point to the base URL of the API you want to test against.

See: https://api.wellcomecollection.org/catalogue/v2/_elasticConfig

To run the works test against the pipeline-prod cluster, you can run the following command:

uv run rank test \
  --content-type=works \
  --cluster=pipeline-prod \
  --query=https://api.wellcomecollection.org/catalogue/v2

Or to target a specific pipeline date:

uv run rank test \
  --content-type=works \
  --pipeline-date=2025-10-02 \
  --query=https://api.wellcomecollection.org/catalogue/v2

rank is also published as a Docker image in our private ECR registry. There’s a helper script to build and push an image tag for you:

./scripts/deploy_image.sh

By default this pushes the tag dev:

• 756629837203.dkr.ecr.eu-west-1.amazonaws.com/weco/rank:dev

To push a different tag:

./scripts/deploy_image.sh --tag my-tag

Other projects run rank in CI using the image tag latest:

• 756629837203.dkr.ecr.eu-west-1.amazonaws.com/weco/rank:latest

Only push latest when you intentionally want to update what those projects will run in CI.

• Docker (with buildx available)
• AWS credentials with access to the ECR repositories
• The AWS CLI profile catalogue-developer (as referenced in scripts/deploy_image.sh)