feat(consensus): add prefix_hash to data tx and fold into ledger tx_root#1450
feat(consensus): add prefix_hash to data tx and fold into ledger tx_root#1450DanMacDonald wants to merge 2 commits into
Conversation
Adds a signed per-tx `prefix_hash` and folds `hash_all_sha256([data_root,
prefix_hash])` into each tx_root leaf, so an indexer holding only the
block-signature-sealed tx_root can trust every tx's prefix_hash without
verifying individual tx signatures. Softfork (no block-header change;
empty ledgers still fold to H256::zero()).
- rename header_size -> prefix_size; add prefix_hash: H256
- block validation recomputes tx_root and rejects TxRootMismatch
- PoA data-ledger branch recovers the owning tx's data_root (the folded
leaf no longer yields it) and binds it via the fold
- storage retrieval recovers data_root from a new submodule binding
table (tx_path_hash -> {data_root, prefix_hash}), verified by the fold
- block.rs: fold load-bearing, compute_tx_root == merklize root, indexer reconstruction, and fold == hash_all_sha256 (gateway-compat guard) - signature.rs: prefix_size/prefix_hash are covered by the tx signature - chain-tests: a block whose tx_root doesn't match its txs is rejected with TxRootMismatch
📝 WalkthroughWalkthroughReplaces Changesprefix_hash field + PoA tx-root/data-root binding
Sequence Diagram(s)sequenceDiagram
participant prevalidate_block
participant poa_is_valid
participant load_owning_tx_for_poa
participant DataTransactionLedger
participant DB
rect rgba(135, 206, 250, 0.5)
Note over prevalidate_block: tx_root recomputation check
prevalidate_block->>DataTransactionLedger: compute_tx_root(included_txs)
DataTransactionLedger-->>prevalidate_block: recomputed_root
prevalidate_block->>prevalidate_block: compare vs header.tx_root → TxRootMismatch?
end
rect rgba(144, 238, 144, 0.5)
Note over poa_is_valid: PoA tx-path leaf binding
poa_is_valid->>DB: get_data_poa_bounds (returns BlockBounds + owning_block_hash)
poa_is_valid->>load_owning_tx_for_poa: owning_block_hash, byte_range
load_owning_tx_for_poa->>DB: fetch DataTransactionHeader
DB-->>poa_is_valid: owning_tx
poa_is_valid->>DataTransactionLedger: tx_root_leaf_value(owning_tx)
DataTransactionLedger-->>poa_is_valid: expected_leaf (fold data_root+prefix_hash)
poa_is_valid->>poa_is_valid: compare vs tx_path leaf → PoaTxRootLeafMismatch?
poa_is_valid->>poa_is_valid: verify data chunk proof against owning_tx.data_root
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/actors/src/block_validation.rs`:
- Around line 3231-3236: The function ledger_tx_ids_in currently uses map()
which collapses a missing owning ledger into None, and callers use
unwrap_or_default() to silently treat this as an empty tx list, causing the
error to be misattributed as a PoAChunkOffsetOutOfTxBounds error instead of
BlockBoundsLookupError. Change the return type from Option<Vec<H256>> to a
Result that returns BlockBoundsLookupError when the ledger is not found, and
return a borrowed slice instead of cloning the full tx-id list to avoid
unnecessary cloning on every migrated PoA lookup. Update all call sites that
currently use unwrap_or_default() to properly handle the Result type and
propagate the BlockBoundsLookupError accordingly.
In `@crates/tooling/multiversion-tests/src/data_tx.rs`:
- Around line 183-184: The issue is that when tx.header.prefix_size is assigned
to 64 (when tx_build.keep_default does not include "prefix_size"), the
tx.header.prefix_hash field is never recomputed to match. Since prefix_hash must
be SHA-256(first prefix_size bytes), modifying prefix_size without updating
prefix_hash creates an inconsistency between the two fields that both get signed
in the subsequent sign_transaction call, causing compat failures. To fix this,
after assigning tx.header.prefix_size = 64, recompute tx.header.prefix_hash by
computing the SHA-256 hash of the first 64 bytes of the transaction data before
calling sign_transaction, or alternatively prevent the post-creation mutation by
computing both prefix_size and prefix_hash at transaction creation time instead.
In `@crates/types/src/block.rs`:
- Around line 1927-1940: The test function
fold_tx_root_leaf_matches_hash_all_sha256 uses a manual loop to iterate through
test cases instead of using the rstest framework. Convert this to a
parameterized test by applying the #[rstest] macro to the function, adding each
test case as a #[case] attribute with the data_root and prefix_hash parameters,
and replacing the loop body with a single assertion that uses the parameterized
inputs. This will provide per-case reporting and align with the repository's
test conventions.
- Around line 728-736: The folded_leaves function uses an unchecked cast of
h.data_size (a u64) to usize with the as keyword, which silently truncates on
32-bit platforms. Replace the unsafe as usize cast in the tx_size field
assignment with usize::try_from(h.data_size).expect(...) to perform a checked
conversion that will panic if the value cannot be represented as a usize,
preventing silent data loss.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: a0f2589e-64c1-44fc-8adb-e777c8a4c407
📒 Files selected for processing (18)
crates/actors/src/block_validation.rscrates/actors/src/mempool_service.rscrates/chain-tests/src/block_production/block_validation.rscrates/chain-tests/src/validation/mempool_ingress_proof_dedup.rscrates/chain-tests/src/validation/mod.rscrates/database/src/migration.rscrates/database/src/submodule/db.rscrates/database/src/submodule/tables.rscrates/database/src/tables.rscrates/domain/src/models/storage_module.rscrates/p2p/src/wire_types/data_transaction.rscrates/p2p/src/wire_types/test_helpers.rscrates/tooling/multiversion-tests/src/data_tx.rscrates/tooling/multiversion-tests/src/run_config.rscrates/types/src/block.rscrates/types/src/signature.rscrates/types/src/transaction.rsfixtures/gossip_fixtures.json
Summary
Adds a per-transaction
prefix_hash(SHA-256 of the leading prefix bytes of txs' data) and folds it into each data ledger'stx_root, so an indexer or light client holding only the block-signature-sealedtx_rootcan trust every transaction'sprefix_hash— and the canonical tags it commits to — without verifying any individual transaction's signature. A verifier reconstructs the folded root from each tx's(data_root, prefix_hash)and compares it to the signedtx_root.This is a softfork: no block-header field is added and the signed block preimage is unchanged. The only changes are (a) the data-transaction format —
header_sizeis renamed toprefix_sizeand a signedprefix_hash: H256field is added — and (b) howtx_rootis derived. Empty ledgers still fold toH256::zero(), so no existing block'stx_rootchanges meaning. The leaf formula,hash_all_sha256([data_root, prefix_hash]), matches the gateway's, so cross-implementationtx_rootreconstruction holds.Design notes
Folding changes each
tx_rootleaf fromdata_roottohash_all_sha256([data_root, prefix_hash]). Becausevalidate_pathreturns the leaf's stored hash, atx_pathproof no longer yields the rawdata_root— and the node previously readdata_rootstraight out of that leaf in three places. Each now recovers the realdata_rootfrom an independent source and re-verifies it against the proof leaf via the same fold, so the two can never silently diverge:tx_rootenforcement (prevalidate_block) — recomputes each ledger'stx_rootfrom the included txs' folded leaves and rejectsTxRootMismatch. This is what makes the block signature transitively authenticate everyprefix_hash.poa_is_valid) — recovers the recall chunk's owning transaction (in-memory block_tree for tip blocks, else the DB), binds thetx_pathleaf tofold(data_root, prefix_hash), and validates thedata_pathagainst the realdata_root.get_chunk_by_offset/get_chunk_metadata) — recoversdata_rootfrom a new per-submodule tableTxLeafBindingByTxPathHash(tx_path_hash → {data_root, prefix_hash}), cross-checked against the proof leaf via the fold. (Additive table — created empty on open, no data migration.)The fold lives in one place —
DataTransactionLedger::fold_tx_root_leaf— and every consumer (block production, validation recompute, the PoA binding, and the storage check) delegates to it.Test plan
cargo clippy --workspace --testsandcargo fmt --allclean.prefix_hashchangestx_root);compute_tx_root==merklize_tx_rootroot; an indexer can reconstructtx_rootfrom(data_root, prefix_hash); the fold is byte-identical tohash_all_sha256(gateway-compat guard);prefix_size/prefix_hashare covered by the tx signature.tx_rootdoesn't match its txs is rejected withTxRootMismatch; valid blocks pass.data_poa_at_tip_validates_via_block_tree_fallback) and migration-depth-2 (spiky_heavy_mine_ten_blocks_with_migration_depth_two); a tampered leaf is rejected.spiky_heavy_api_end_to_end_test_32b) and the full p2p gossip-fixture suite pass.🤖 Generated with Claude Code