Summary
The Postgres adapter reads events with SELECT * and hands the rows straight to class_row(StoredEvent[...]). Any column in the events table that the installed version of StoredEvent does not know about becomes an unexpected keyword argument, and the read fails:
TypeError: StoredEvent.__init__() got an unexpected keyword argument 'metadata'
This makes purely additive schema changes breaking at read time, and it makes rolling deployments hard.
Why this matters
Adding metadata in 0.1.12a3 requires a migration on the events table. In a rolling deployment there is a window where the migrated table is live but old tasks still run 0.1.11. Those old tasks read the new column and crash, even though they never asked for it.
We worked around this in an expand/contract sequence of four PRs:
- Add a read shim to the
0.1.11 service so it tolerates the extra column (expand).
- Migrate the table with
metadata JSONB NOT NULL DEFAULT 'null'::jsonb, no service change.
- Upgrade to
0.1.12a3 and delete the shim.
- Migrate again to drop the default, leaving
metadata JSONB NOT NULL (contract).
Steps 1 and 3 exist only because of the SELECT * behaviour. Without them the upgrade would have been two migrations and one version bump.
Suggested fix
Either would remove the problem:
- Select the columns the installed model has, instead of
*. This is the stronger option: the query becomes self-describing and the adapter reads exactly what it can construct.
- Ignore unknown columns when building the
StoredEvent, so extra columns are dropped rather than passed to __init__.
Where
src/logicblocks/event/store/adapters/postgres/adapter.py:309 — select_clause = sql.SQL("SELECT *") in read_last_query
src/logicblocks/event/store/adapters/postgres/adapter.py — the four class_row(StoredEvent[str, JsonValue, JsonValue]) cursors (lines 747, 805, 894, 912)
src/logicblocks/event/persistence/postgres/query.py:109 — Star, used by the scan/query paths
Reproduce
- Run a service on
0.1.11.
- Add a
metadata column to the events table.
- Read any event. The read raises the
TypeError above.
Verified locally.
Summary
The Postgres adapter reads events with
SELECT *and hands the rows straight toclass_row(StoredEvent[...]). Any column in theeventstable that the installed version ofStoredEventdoes not know about becomes an unexpected keyword argument, and the read fails:This makes purely additive schema changes breaking at read time, and it makes rolling deployments hard.
Why this matters
Adding
metadatain0.1.12a3requires a migration on theeventstable. In a rolling deployment there is a window where the migrated table is live but old tasks still run0.1.11. Those old tasks read the new column and crash, even though they never asked for it.We worked around this in an expand/contract sequence of four PRs:
0.1.11service so it tolerates the extra column (expand).metadata JSONB NOT NULL DEFAULT 'null'::jsonb, no service change.0.1.12a3and delete the shim.metadata JSONB NOT NULL(contract).Steps 1 and 3 exist only because of the
SELECT *behaviour. Without them the upgrade would have been two migrations and one version bump.Suggested fix
Either would remove the problem:
*. This is the stronger option: the query becomes self-describing and the adapter reads exactly what it can construct.StoredEvent, so extra columns are dropped rather than passed to__init__.Where
src/logicblocks/event/store/adapters/postgres/adapter.py:309—select_clause = sql.SQL("SELECT *")inread_last_querysrc/logicblocks/event/store/adapters/postgres/adapter.py— the fourclass_row(StoredEvent[str, JsonValue, JsonValue])cursors (lines 747, 805, 894, 912)src/logicblocks/event/persistence/postgres/query.py:109—Star, used by the scan/query pathsReproduce
0.1.11.metadatacolumn to theeventstable.TypeErrorabove.Verified locally.