High-performance Postgres replication, written in Rust.
Embed it as a library or run it as a standalone binary.
A project by Supabase.
Documentation · First Pipeline · Standalone Replicator · Examples · Issues
Note
Supabase ETL is under active development. APIs and setup steps may change before the first stable release.
Supabase ETL is a high-performance Postgres replication engine written in Rust. Embed it in your Rust application or run it as a standalone binary. For each published table, it performs an initial sync of existing rows, then replicates changes to a built-in or custom destination.
This repository contains the framework, destination modules, standalone replicator, examples, and documentation. For the managed product in the Supabase Dashboard—including availability, pricing, and operations—use the Supabase Pipelines documentation.
flowchart LR
Postgres["Postgres publication"] --> Sync["Initial sync"]
Sync --> Replication["Ongoing replication"]
Replication --> Destination["Destination"]
Supabase ETL replicates each table in two phases:
- Initial sync: Copy the existing rows selected by the publication.
- Ongoing replication: Capture subsequent inserts, updates, deletes, and truncates, then deliver those changes as ordered events.
Across both phases, a store persists checkpoints, schemas, destination metadata, and table state so replication can recover safely after a restart. Streaming is a transfer mode that may be used within either phase; it is not a separate replication phase.
| Goal | Documentation |
|---|---|
| Build a working pipeline | First Pipeline |
| Run the standalone process | Standalone Replicator |
| Prepare a source database | Configure Postgres |
| Implement a store or destination | Custom Implementations |
| Understand the runtime | Architecture |
| Browse runnable destinations | etl-examples |
ETL is installed from Git while we prepare for a crates.io release:
[dependencies]
etl = { git = "https://github.com/supabase/etl" }
tokio = { version = "1", features = ["full"] }- Library or standalone binary: Embed the engine in a Rust application or run the ready-made replicator.
- High performance, small footprint: Run one Rust process without Kafka, Flink, Debezium, or another coordination service.
- Postgres-native selection: Use publications to select tables, columns, rows, and operation types.
- Flexible by design: Implement custom destinations and durable stores with typed Rust APIs.
- Recovery-aware: Persist checkpoints and table state for safe restarts and at-least-once delivery.
| Feature | Destination | Status |
|---|---|---|
bigquery |
Google BigQuery | Stable |
clickhouse |
ClickHouse | In progress |
ducklake |
DuckLake | In progress |
snowflake |
Snowflake | In progress |
iceberg |
Apache Iceberg | Deprecated |
BigQuery is the stable, recommended default. See the
Destinations reference
for maturity and limitations, and the
etl-examples guide for runnable examples.
Supabase ETL supports PostgreSQL 14 through 18. PostgreSQL 15 or newer is recommended for column and row publication filters. PostgreSQL 16 or newer is required when the replication connection points at a physical read replica.
The source must use wal_level = logical, and the replication user needs the
REPLICATION role. See Configure Postgres
for the complete setup and production guidance.
See DEVELOPMENT.md for local setup, migrations, formatting,
linting, and tests. The workspace uses Rust 1.95.0 from rust-toolchain.toml.
Contributions are welcome. Before proposing a new destination, start a discussion or issue so maintainership and long-term demand can be evaluated first.
Report suspected vulnerabilities privately according to SECURITY.md.
Apache-2.0. See LICENSE.