Skip to content

Repository files navigation

Portolan catalog template

🌐 Explorar el catálogo (visor web) →

Note (2026-06): aligning to the canonical Portolan spec. A catalog is now a STAC-profile directory (.portolan/ + catalog.json + versions.json + one {collection}/ per dataset), built with portolan-cli (init/add/check) and annotated with the git-backed-catalog (git:) and STAC Iceberg (iceberg:) extensions. See docs-migration-recipe.md for the exact recipe. The config-driven tools/*.py below are the previous generation and are being superseded by portolan-cli + extension injection.**

A GitHub template for a Portolan spatial-data catalog: a git repo that is the source of a catalog, and publishes itself to object storage as a static Apache Iceberg REST catalog (ATTACH from DuckDB / Snowflake) + STAC + OGC API - Records + a human HTML explorer — no server. The git repo holds only the definition (config + STAC + small Iceberg metadata); the data bytes live on the bucket.

Create a new catalog from this template

  1. Use this template → new repo jatorre/portolan-<name>-catalog.
  2. portolan.config.json — copy portolan.config.example.json and fill in publisher, bucket path, public_base, contributions repo. (One file; the tooling + workflows read it.)
  3. datasets/<id>.json — one compact descriptor per dataset (see schema in tools/generate_stac.py). Set representation to iceberg (+ tables), raquet (+ href), or remote-geoparquet (+ href).
  4. data/<ns>/<table>/metadata/ — the Apache Iceberg metadata (v1.metadata.json + manifests), with URLs pointing at your public_base. The parquet bytes are not committed (git-ignored); upload them to the bucket at …/data/<ns>/<table>/data/.
  5. python tools/generate_stac.py → writes catalog.json, items/, records/.
  6. python tools/validate.py → checks it all (run before committing).
  7. Add repo secrets UPCLOUD_ACCESS_KEY / UPCLOUD_SECRET_KEY; push. The Action publishes to the bucket. (Or run python tools/publish.py locally with an mc alias configured.)
  8. Add the catalog as a child of the federation catalog-of-catalogs.

Layout

portolan.config.json          publisher + bucket config (the only per-repo config)
datasets/<id>.json            compact dataset descriptors (hand-authored)
catalog.json, items/, records/  STAC + OGC API - Records — GENERATED by generate_stac.py
data/<ns>/<table>/metadata/   Iceberg metadata — IN GIT (points at parquet by bucket URL)
data/**/data/*.parquet        data bytes — ON THE BUCKET ONLY (git-ignored)
tools/generate_stac.py        descriptors + config -> STAC + Records
tools/publish.py              mirror metadata + generate Iceberg REST catalog + HTML -> bucket
tools/validate.py             PR gate (schema, URLs, bucket data exists, bblock)
bblock/portolan-record/       OGC Building Block for the portolan:* extension
.github/workflows/            publish (on merge) + validate (on PR)
.github/ISSUE_TEMPLATE/       usage-report (opt-in intent telemetry)

Principles (don't regress)

  • Git = definition; bucket = data + generated artifacts. Never commit parquet.
  • Anonymous + static + open. Authenticated/private data is a separate, commercial concern — not part of this open template.
  • Standards: STAC + OGC API - Records (both GeoJSON; both OGC), GeoParquet, Apache Iceberg. Query is the engine's native SQL — no custom query API.

Generated catalogs are read four ways from the bucket: ATTACH, iceberg_scan, STAC/Records, and direct parquet download.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages