Skip to content

Tags: Mu-L/marmot

Tags

v2.9.2-beta

Toggle v2.9.2-beta's commit message
Fix: DDL streaming replication - persist DDL to CDC storage

Previously, DDL statements (CREATE/DROP/ALTER TABLE, CREATE/DROP INDEX)
were stored only in write intents which were cleaned up after commit.
This caused DDL transactions to stream with empty statements, breaking
replication to followers.

Changes:
- Add OpTypeDDL constant for DDL operations in CDC
- Add DDLSQL field to EncodedCapturedRow for DDL storage
- Add writeDDLToCDC() to persist DDL to /cdc/raw/ during commit
- Update sendChangeEvent() to handle OpTypeDDL with DDLChange payload
- Fix gap detection in stream_client.go - txn_ids are HLC-based
  (nanoseconds), not sequential counters, so comparing against
  transaction count threshold caused infinite snapshot loops

DDL replication flow:
1. Leader commit -> writeDDLToCDC() writes to /cdc/raw/
2. Streaming reads from /cdc/raw/ via IterateCapturedRows()
3. sendChangeEvent() creates DDLChange payload for OpTypeDDL
4. Replica applies DDL via RewriteDDLForIdempotency()

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

v2.9.1-beta

Toggle v2.9.1-beta's commit message
Fix: CREATE DATABASE via write forwarding fails with syntax error

The forward handler created ConnectionSession with TranspilationEnabled=false
(the Go default). This caused the coordinator to skip transpilation when
parsing forwarded SQL, which resulted in:
- CREATE DATABASE being classified as StatementDDL instead of StatementCreateDatabase
- Empty database name field (defaulting to session's current database)
- SQLite receiving "CREATE DATABASE x" which it doesn't support

The fix enables transpilation for forwarded queries, ensuring proper statement
type detection for CREATE DATABASE, DROP DATABASE, and other special statements.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

v2.9.0-beta

Toggle v2.9.0-beta's commit message
Feat: Add write forwarding from read-only replicas to leader

Implements transparent write forwarding allowing read-only replicas to
forward mutations to the leader node. Key features:

- ForwardQuery gRPC endpoint for forwarding mutations and transactions
- Transaction buffering with BEGIN/COMMIT/ROLLBACK support
- Msgpack serialization for query parameters
- TxnWaitQueue for waiting on transaction replication
- DDL statements automatically wait for replication before returning
- Multi-statement SQL support (e.g., CREATE TABLE with KEY generates
  both CREATE TABLE and CREATE INDEX)
- Session state tracking for forwarded transactions

Bug fixes included:
- Transaction state synced to replica (ForwardedTxnActive field)
- Multi-statement SQL properly split on leader
- DDL schema visible immediately after forward completes
- Fixed gossip integration test flakiness (gRPC warmup)

New files:
- grpc/forward_handler.go - Handles forwarded queries on leader
- grpc/forward_session.go - Session management for forwarded connections
- grpc/forward_params.go - Msgpack param serialization
- replica/txn_wait_queue.go - Wait for transaction replication

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

v2.8.0

Toggle v2.8.0's commit message
v2.8.0: Add file-based logging with rotation and daemon mode

v2.7.2

Toggle v2.7.2's commit message
Feat: Wire up new admin endpoints in dashboard UI

- Overview tab: Added cluster health banner with quorum status
- Overview tab: Added databases table with file size, txn counts
- Cluster tab: Extended with role, is_local, is_seed flags
- New Replication tab: Shows peer replication lag per database
- Transactions tab: Added aborted/all status filters and abort button
- Database dropdown now dynamically populated from /admin/databases

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

v2.7.1

Toggle v2.7.1's commit message
Fix: Replica snapshot verification and codebase cleanup

Replica Snapshot Fix:
- Filter out __marmot_system from snapshot file verification list
- Replicas maintain their own independent system database
- Master's StreamSnapshot already excludes system DB, but GetSnapshotInfo
  included it, causing verification mismatch

Staticcheck Cleanup:
- Remove unused functions: getConflictWindow, handleCreateDatabase,
  handleDropDatabase, syncPeer, parseIntentByTxnKey, fetchRow,
  readPIDFile, waitForRowCount, createTxnIDOrderingTestStore
- Remove unused fields: db in PendingLocalExecution, inUpdateSet in
  deterministicChecker
- Remove unused constants and delete db/cleanup.go (entirely dead code)
- Fix unused metaStore assignment in grpc/server.go

Deprecated API Migration:
- Replace grpc.DialContext with grpc.NewClient (4 locations in catch_up.go)
- Replace grpc.Dial with grpc.NewClient in gossip_integration_test.go
- Replace grpc.DialContext with grpc.NewClient in stream_client.go
- Remove grpc.WithBlock() option (not needed with NewClient)

Code Quality:
- Replace single-case select with simple channel receive
- Remove redundant nil checks before len() on slices
- Fix -0.0 literal (same as 0.0 in Go)
- Remove unnecessary type assertion to same type
- Fix unused append result in database_discovery_test.go
- Run go mod tidy to remove unused dependencies

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

v2.6.0

Toggle v2.6.0's commit message
Fix: CDC Type override breaking UPSERT replication

Remove two lines that incorrectly override statement Type after
ConvertToStatement() correctly computed it from CDC data.

Root cause: For UPSERT (INSERT OR REPLACE) on existing rows, SQLite
fires two hooks: DELETE (OldValues only) then INSERT (NewValues only).
ConvertToStatement() correctly sets Type based on which values exist.

The bug: Lines 441 and 897 were overriding Type back to REPLACE,
causing the batch committer to call ApplyCDCInsert with nil NewValues
for DELETE entries, resulting in "no values to insert" errors.

Symptoms:
- "partial commit: got 0 remote commit acks" errors
- "ApplyCDCInsert: no values to insert" in batch committer
- 99%+ of errors were UPSERT-related (9181/9204 in load test)

The fix preserves SQL and Database for debugging but lets
ConvertToStatement() determine the correct Type from CDC data.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

v2.4.2

Toggle v2.4.2's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Merge pull request maxpert#141 Improving Performance

Multiple performance improvements

v2.4.1

Toggle v2.4.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Merge pull request maxpert#139 from maxpert/feature/sqlite-batch-comm…

…itter

Introducing ability to batch commit changes at storage level

v2.4.0

Toggle v2.4.0's commit message
Merge feature/grouped-2pc-v2: StatementCode consolidation and securit…

…y fixes