Skip to content

About

Partition-aware online schema evolution for PostgreSQL: build indexes across every partition without blocking writes, from a Liquibase changeset or a standalone CLI.

Topics

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

PartitionCTL

CI License

Partition-aware online schema evolution for PostgreSQL.

CREATE INDEX CONCURRENTLY is rejected outright on a partitioned parent table:

ERROR:  cannot create index on partitioned table "orders" concurrently

The documented workaround is a per-partition choreography — build the parent index ON ONLY, build one index concurrently per leaf, then ATTACH each one, at which point PostgreSQL validates the parent on the final attach. It has to be generated from the live catalog, because the statement list depends on how many partitions exist today. On a table with a few hundred partitions that is several hundred order-sensitive statements, any one of which can fail and leave the tree in a state the next run has to understand.

This repository does that for you, two different ways.


Two independent products. Pick one.

They share no code, do not interoperate, and neither is a wrapper around the other.

1. The Liquibase extension — liquibase-partitionctl/

One Maven dependency and one changeset. The whole operation happens inside liquibase update: no binary to install, no second set of credentials, no step outside your existing migration pipeline.

→ liquibase-partitionctl/README.md — quickstart, full attribute reference, troubleshooting and limitations.

It is on Maven Central, so there is no repository to declare — one dependency, in the liquibase-maven-plugin's own <dependencies>:

<!-- inside the liquibase-maven-plugin's OWN <dependencies>, not your project's -->
<dependency>
  <groupId>io.github.atulsinha87</groupId>
  <artifactId>liquibase-partitionctl</artifactId>
  <version>0.1.4</version>
</dependency>

The plugin's <dependencies>, not your project's, is the one thing worth getting right: in your project's dependencies the extension is invisible to Liquibase and every <ext:...> tag fails with "no declaration can be found".

examples/liquibase/ is a complete, runnable consumer of exactly this coordinate — make lb-adopter runs it against a throwaway 12-partition database and verifies the result from pg_catalog.

Building from source works too, and installs the same coordinate:

git clone https://github.com/atulsinha87/partitionctl.git
cd partitionctl && ./mvnw clean install

2. The Go CLI — engine/, operations/, adapters/, cmd/

A standalone plan → execute → verify loop with a reviewable plan artifact, a state store and resume. Use it when you want the migration to be an artifact you inspect and approve before it runs, independent of Liquibase.

make build
make db-reset                    # throwaway PostgreSQL 17, 12 partitions, ~1.05M rows

# The DSN carries the password, so it is never a flag (a flag would put it on argv,
# visible in ps). Point this at your own database to use the CLI for real.
export PARTITIONCTL_DSN='postgres://postgres:pw@localhost:5432/postgres?sslmode=disable'

build/partitionctl plan -spec examples/local/orders-idx.json -o build/migration.plan
build/partitionctl render build/migration.plan     # the SQL runbook, offline
build/partitionctl execute build/migration.plan
build/partitionctl verify -end-state build/migration.plan

Without PARTITIONCTL_DSN the discrete flags apply, and sslmode defaults to require — so a server without TLS refuses the connection. Pass -sslmode disable for a local throwaway database. make demo-all runs this whole sequence with every command echoed.


What it does, and what it costs

Operation Blocking?
Create an index over every partition No. One CREATE INDEX CONCURRENTLY per leaf, attached as it goes
Reindex every partition No. REINDEX INDEX CONCURRENTLY per leaf
Drop a partitioned index Yes, unavoidably. See below

Dropping is the one operation that is not online. DROP INDEX on a partitioned parent takes AccessExclusiveLock on the parent and every leaf, simultaneously. PostgreSQL offers no ALTER INDEX … DETACH PARTITION, so attached children cannot be detached and dropped one at a time — the inverse of ATTACH does not exist. Both tools therefore refuse to drop without an explicit confirmation flag, and say what the lock will cost before they take it.

Requirements

  • PostgreSQL with declarative partitioning. Verified on 14.23 and 17.10.
  • Liquibase extension: Java 8 or newer at runtime; liquibase-core 4.x (provided, so the jar contains zero Liquibase classes and no transitive dependencies at all). Build with JDK 17.
  • Go CLI: Go 1.22 or newer.
  • Single-level partitioning. Sub-partitioned tables are refused, not silently mishandled — ALTER INDEX … ATTACH PARTITION only accepts an index on a direct partition.

Status

v0.1.4, published to Maven Central. Both products have been exercised against a live PostgreSQL 17.10 with 12 partitions and 1.2M rows, through the whole create → gate → reindex → drop cycle, with every verdict read from pg_catalog rather than from either tool's own log — including SIGKILL mid-flight and a lock_timeout abort, both of which resumed and finished.

Known limitations

The Liquibase extension's limitations are documented plainly in liquibase-partitionctl/README.md. They are real constraints of PostgreSQL and Liquibase, not a to-do list.

The Go CLI has one behaviour worth knowing about at PostgreSQL's 63-byte identifier limit. When a _ccnew/_ccold leftover fills all 63 bytes, PostgreSQL has truncated the base name to make room for the suffix, and the bytes that would tell two similarly-named indexes apart are gone. Where more than one index on the partition could have produced the leftover, the CLI refuses to act rather than guess — it will not take an ownership marker off an object that might not be the right one. Resolve it by hand and re-run. This was #2, fixed; the refusal is the deliberate end state, not a remaining bug.

Verifying a checkout

gofmt -l . && go vet ./... && go build ./... && go test ./... -count=1
mvn clean install

Live database work needs Docker:

make db-reset && make demo-all     # the Go CLI, end to end
make lb-e2e                        # the extension: create, gate, reindex, drop

On macOS the Liquibase Maven plugin needs MAVEN_OPTS=-Duser.timezone=UTC, or the JDBC connect fails on a non-UTC system timezone. The harness scripts set it.

Contributing

See CONTRIBUTING.md. The short version: run it, and say what it printed. A green test suite is treated as weak evidence here — this project has shipped a feature broken for every possible input while 579 tests passed, and two releases that built cleanly and failed on publish. Pull requests should keep "written" and "verified" visibly separate.

CI runs the two offline gates above on every pull request. The Docker-backed gates are run locally; CONTRIBUTING.md says which one matches your change.

Licence

Apache License 2.0 — see LICENSE and NOTICE.

About

Partition-aware online schema evolution for PostgreSQL: build indexes across every partition without blocking writes, from a Liquibase changeset or a standalone CLI.

Topics

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages