feat(proto): create v2 execution API #1962

ethanoroshiba · 2025-02-11T19:04:39Z

Summary

Creates v2 execution API to support upcoming v2 changes to Conductor and Geth.

Background

These changes are meant to support the upcoming v2 upgrades to Conductor and astria-geth, which will facilitate the ability to have rollup forks, driven by the need to migrate Forma to Mainnet.

Changes from `v1`

Removed GenesisInfo and associated RPC.
Removed BatchGetBlocks RPC and associated types.
Removed GetCommitmentState RPC.
Introduced ExecutionSession and associated RPC CreateExecutionSession:
- An ExecutionSession is the execution client-side's current session, which starts at a specific block number and can end at one as well.
- Upon startup and after each restart (if stop numbers are specified), CreateExecutionSession should be called, to which the server should respond with the current session, containing an ID for the session, the CommitmentState, and ExecutionSessionParams.
Added ExecutionSessionParams, which contains the necessary configuration for the execution client to run for the entirety of the session, consisting of the following:
- rollup_id
- rollup_start_block_number, the first block to be executed against the rollup
- rollup_end_block_number, the final block to be executed against the rollup before ending the current execution session.
- sequencer_chain_id
- sequencer_start_block_height, the sequencer height to map to rollup_start_block_number.
- celestia_chain_id
- celestia_max_search_height_look_ahead (formerly celestia_block_variance)
Added session_id to ExecuteBlockRequest and UpdateCommitmentStateRequest, so that the server can validate the calls are for the correct execution session.
Changed Block to ExecutedBlockMetadata and made corresponding service changes.
Added ExecuteBlockResponse to follow AIP.
Changed CommitmentState.base_celestia_height to lowest_celestia_search_height
Represented all hashes as Strings so that they are displayed in whichever format the hashes are in, as opposed to Base64.
Separated all top-level proto entities into separate files (per 1-1-1 best practice)

Testing

No testing needed.

Changelogs

Protos have no changelog

ethanoroshiba · 2025-02-11T19:06:46Z

proto/executionapis/astria/execution/v2/execution.proto

+  astria.primitive.v1.RollupId rollup_id = 1;
+  // The first block height on the sequencer chain to use for rollup transactions.
+  // This is mapped to `rollup_first_block_number`.
+  uint32 sequencer_first_block_height = 2;


I know we did away with sequencer_start_block_height in favor of sequencer_start_height before, but somehow sequencer_first_height feels like it carries less information than sequencer_first_block_height to me. Happy to change, though

proto/executionapis/astria/execution/v2/execution.proto

…length, make uint32s uint64s

SuperFluffy

Left some notes for discussion on the need for rollup_halt_at_stop_number.

Going down this rabbit hole - I wonder if we can avoid setting rollup_stop_block_number entirely in favor of using gRPC error codes?

For example, OUT_OF_RANGE or RESOURCE_EXHAUSTED might be valid error codes that can trigger conductor to re-fetch the genesis and/or commitment states.

proto/executionapis/astria/execution/v2/execution.proto

joroshiba

Approved, but noting there is one extra file here were we should delete.

Either as a part of this PR or as follow up can we get a execution api v2 spec doc?

proto/executionapis/astria/execution/v2/get_commitment_state_request.proto

ethanoroshiba · 2025-02-19T21:12:40Z

Either as a part of this PR or as follow up can we get a execution api v2 spec doc?

Noting that I have made a followup issue here for the spec: #1986. Prefer to get this landed ASAP

SuperFluffy

This looks good overall.

I only have a bunch of naming and field ordering requests (important to get this right because future changes would be breaking).

proto/executionapis/astria/execution/v2/block.proto

proto/executionapis/astria/execution/v2/block_identifier.proto

proto/executionapis/astria/execution/v2/execution_config.proto

SuperFluffy · 2025-02-20T13:48:06Z

proto/executionapis/astria/execution/v2/execution_config.proto

+  // Celestia blobs).
+  string celestia_chain_id = 6;
+  // The allowed variance in celestia for sequencer blocks to have been posted.
+  uint64 celestia_block_variance = 7;


@joroshiba I think we should define a fixed variance here and not perform a calculation inside conductor. Right now it's 6x this value, but I don't see a reason to not just set it here without extra conductor shenanigans.

Co-authored-by: Superfluffy <janis@astria.org>

SuperFluffy

I have some thoughts on the block fields in CommitmentState. I think we should define bespoke SoftRollupBlock and FirmRollupBlock (or alternatively, SoftCommitmentState and FirmCommitmentState, which we will populate from the executed RollupBlock conductor side).

proto/executionapis/astria/execution/v2/commitment_state.proto

SuperFluffy

Left some comments on the response types of the RPCs. We are sometimes relatively strict about naming these things (for example in the auctioneer protos), but here we are loser.

Should we enforce a *Request -> *Response convention everywhere (where for example GetBlock responds with a GetBlockResponse { rollup_block: RollupBlock } (so that the block is just a field of the response)?

proto/executionapis/astria/execution/v2/execution_service.proto

proto/executionapis/astria/execution/v2/rollup_block.proto

proto/executionapis/astria/execution/v2/execution_service.proto

proto/executionapis/astria/execution/v2/commitment_state.proto

proto/executionapis/astria/execution/v2/execution_service.proto

proto/executionapis/astria/execution/v2/rollup_block.proto

proto/executionapis/astria/execution/v2/rollup_block_identifier.proto

proto/executionapis/astria/execution/v2/execution_session_parameters.proto

joroshiba

LGTM

SuperFluffy

We need to fix the type of the ExecuteBlock.prev_block_hash request to match the ExecutedBlockMetadata.

proto/executionapis/astria/execution/v2/execute_block_request.proto

## Summary Updates Conductor to use v2 Execution API, including use of new "execution sessions". ## Background These changes are necessary to support migrating Forma to mainnet. Migrating Forma to Astria's main net requires introducing stop and start logic into the Conductor. A migration using these changes could look like this: 1. Running against a fresh rollup node, Conductor starts and initiates a new execution session via `astria.execution.v2.CreateExecutionSession`, which contains `rollup_start_block_number = 1`, and `rollup_end_block_number = 1000`. 2. Conductor executes (syncs) the rollup node from rollup block numbers 1 to 1000 (inclusive) 3. Upon executing rollup number 1000, conductor resets its inner state and initiates a new execution session via `astria.execution.v2.CreateExecutionSession`, with new information from which it will continue executing. For cases where `rollup_end_block_number` is unset (or equivalently set to 0), conductor will never restart as there is no stop condition. For conductors running in firm-only or in soft-and-firm modes, only the firm-executed block triggers a restart (in soft-and-firm mode, upon soft-executing at the rollup stop number no blocks will be executed past it). For conductors running in soft-only mode the soft-executed block triggers the restart. If the rollup chain is halted, it should respond with `tonic::Code::FailedPrecondition` to `CreateExecutionSession`, which will cause the Conductor to shut down upon attempting a restart. The aforementioned rollup changes to be made in `astria-geth` necessitate changing our current charts config for the following. "Forks" now describes a set of instructions for when and how to stop and restart the EVM at different heights, such that changes (such as network migration) can be made. ```yaml config: # < non astria-prefixed items... > AstriaOverrideGenesisExtraData: AstriaRollupName: AstriaForks: [] # - < fork name >: # height: "" # halt: "" # snapshotChecksum: "" # extraDataOverride: "" # feeCollector: "" # EIP1559Params: {} # sequencer: # chainId: "" # addressPrefix: "" # startHeight: "" # celestia: # chainId: "" # startHeight: "" # searchHeightMaxLookAhead: "" # bridgeAddresses: [] # astriaOracleCallerAddress: "" # astriaOracleContractAddress: "" ``` ## Changes ### In `astria-core` - Replaced v1 execution domain types with v2 execution domain types: - `ExecutionSession` - `ExecutionSessionParameters` - `ExecutedBlockMetadata` - `CommitmentState` (name unchanged, but fields are) - For more information on API changes, see #1962 ### In `astria-auctioneer` - Updated execution types to v2 ### In `astria-conductor` - Changed sequencer-to-rollup-height calculation to start from `rollup_start_block_number` in `ExecutionSessionParameters`, so that Conductor doesn't start from rollup height 1 again after restarting. - Removed `ASTRIA_CONDUCTOR_SEQUENCER_CHAIN_ID` and `ASTRIA_CONDUCTOR_CELESTIA_CHAIN_ID` env vars from Conductor (chain ID checks will now be performed against the values retrieved from the genesis info) - Added restart logic for when the stop block number is reached: Conductor will restart at the stop height and initiate a new execution session - Removed multiplication of Celestia variance. ### In `charts` - Added `astriaForks` to EVM rollup config, to reflect changes in `astria-geth`: https://github.com/astriaorg/astria-geth/pull/59/files. - Add just command for deploying the EVM rollup restart test, which functions the same as our regular smoke test with a restart at sequencer height 20. ## Testing - Added miscellaneous unit tests to ensure heights are correctly calculated and whether to restart is successfully determined. - Added blackbox tests to ensure conductor correctly restarts in each mode, and continues to execute at the correct height after initiating a new execution session. - Added EVM rollup restart test which restarts EVM and Conductor at sequencer height 20 during the regular smoke test. ## Changelogs Changelogs updated ## Breaking Changelist - Domain v1 execution types have been almost entirely replaced - Conductor config is not backwards compatible, i.e. old conductor will work with new config but not vice versa. - Conductor now required v2 execution API ## Related Issues closes #1994 closes #2003 --------- Co-authored-by: Richard Janis Goldschmidt <github@aberrat.io>

## Description  Updates the Execution API docs to reflect latest updates to v2: astriaorg/astria#2006 and astriaorg/astria#1962 ## Motivation and Context   The v2 Execution API starkly contrasts to v1, so changing the documentation seemed appropriate. ## Screenshots (if appropriate):  ## Types of changes  - [x] Edits to existing documentation - [ ] Changing documentation structure (relocating existing files, ensure redirects exist) - [ ] Stylistic changes (provide screenshots above)

## Summary Updated the `execution-api` spec to match the new v2 changes introduced in #1962. ## Background v2 is a decent lift from v1, which warranted a new spec. Additionally, instead of leaving the v1 as well, I've simply replaced it with the v2, since the old spec didn't accurately reflect v1 design anymore. ## Changes - Updated `execution API` spec. ## Testing No tests needed. ## Changelogs No updates required. ## Related Issues closes #1986

## Summary Updated the `execution-api` spec to match the new v2 changes introduced in astriaorg/astria#1962. ## Background v2 is a decent lift from v1, which warranted a new spec. Additionally, instead of leaving the v1 as well, I've simply replaced it with the v2, since the old spec didn't accurately reflect v1 design anymore. ## Changes - Updated `execution API` spec. ## Testing No tests needed. ## Changelogs No updates required. ## Related Issues closes #1986

Make v2 execution API and compile

f7813b2

ethanoroshiba added the proto pertaining to the Astria Protobuf spec label Feb 11, 2025

ethanoroshiba requested a review from joroshiba February 11, 2025 19:04

ethanoroshiba temporarily deployed to BUF February 11, 2025 19:05 — with GitHub Actions Inactive

update comments

42457e9

ethanoroshiba requested a review from SuperFluffy February 11, 2025 19:05

ethanoroshiba temporarily deployed to BUF February 11, 2025 19:05 — with GitHub Actions Inactive

ethanoroshiba commented Feb 11, 2025

View reviewed changes

ethanoroshiba marked this pull request as ready for review February 11, 2025 19:13

ethanoroshiba requested review from a team and Fraser999 as code owners February 11, 2025 19:13

joroshiba reviewed Feb 12, 2025

View reviewed changes

group related fields in 'SequencerInfo', format comments to ~80 line …

746e455

…length, make uint32s uint64s

ethanoroshiba temporarily deployed to BUF February 12, 2025 20:52 — with GitHub Actions Inactive

SuperFluffy reviewed Feb 14, 2025

View reviewed changes

joroshiba and others added 2 commits February 18, 2025 13:23

using sessions

7d47c50

remove commitment level, use 1-1-1 standard

b0fe683

ethanoroshiba temporarily deployed to BUF February 18, 2025 19:39 — with GitHub Actions Inactive

requested changes from @SuperFluffy

2ac64af

ethanoroshiba temporarily deployed to BUF February 18, 2025 19:45 — with GitHub Actions Inactive

joroshiba approved these changes Feb 19, 2025

View reviewed changes

proto/executionapis/astria/execution/v2/get_commitment_state_request.proto Outdated Show resolved Hide resolved

remove unused file

1f3d1c5

ethanoroshiba temporarily deployed to BUF February 19, 2025 21:09 — with GitHub Actions Inactive

SuperFluffy reviewed Feb 20, 2025

View reviewed changes

requested changes

ebeaed9

Co-authored-by: Superfluffy <janis@astria.org>

ethanoroshiba temporarily deployed to BUF February 20, 2025 14:55 — with GitHub Actions Inactive

update 'celestia_height'

0c2992d

ethanoroshiba temporarily deployed to BUF February 20, 2025 19:40 — with GitHub Actions Inactive

SuperFluffy reviewed Feb 20, 2025

View reviewed changes

SuperFluffy mentioned this pull request Feb 25, 2025

chore(docs): v2 execution API spec #1995

Merged

SuperFluffy reviewed Feb 25, 2025

View reviewed changes

joroshiba reviewed Feb 25, 2025

View reviewed changes

naming updates

738199a

ethanoroshiba temporarily deployed to BUF February 25, 2025 20:04 — with GitHub Actions Inactive

slight verbiage change

60fd068

ethanoroshiba temporarily deployed to BUF February 25, 2025 20:09 — with GitHub Actions Inactive

ethanoroshiba requested review from SuperFluffy, joroshiba and mycodecrafting February 25, 2025 20:13

joroshiba approved these changes Feb 26, 2025

View reviewed changes

SuperFluffy reviewed Feb 26, 2025

View reviewed changes

proto/executionapis/astria/execution/v2/execute_block_request.proto Outdated Show resolved Hide resolved

proto/executionapis/astria/execution/v2/execute_block_request.proto Outdated Show resolved Hide resolved

requested changes

25c741f

ethanoroshiba temporarily deployed to BUF February 26, 2025 17:36 — with GitHub Actions Inactive

SuperFluffy approved these changes Feb 26, 2025

View reviewed changes

ethanoroshiba added this pull request to the merge queue Feb 26, 2025

Merged via the queue into main with commit 94e8d14 Feb 26, 2025
48 checks passed

ethanoroshiba deleted the eoroshiba/v2-execution-api branch February 26, 2025 17:50

ethanoroshiba linked an issue Feb 26, 2025 that may be closed by this pull request

Create V2 Execution API to support Forma migration #1993

Closed

ethanoroshiba mentioned this pull request Feb 27, 2025

feat(conductor)!: implement v2 execution API #2006

Merged

ethanoroshiba mentioned this pull request Apr 23, 2025

chore: update execution API docs astriaorg/docs#124

Merged

3 tasks

feat(proto): create v2 execution API #1962

feat(proto): create v2 execution API #1962

Uh oh!

Conversation

ethanoroshiba commented Feb 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Background

Changes from v1

Testing

Changelogs

Uh oh!

ethanoroshiba Feb 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SuperFluffy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

joroshiba left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ethanoroshiba commented Feb 19, 2025

Uh oh!

SuperFluffy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SuperFluffy Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

SuperFluffy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SuperFluffy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

joroshiba left a comment

Choose a reason for hiding this comment

Uh oh!

SuperFluffy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ethanoroshiba commented Feb 11, 2025 •

edited

Loading

Changes from `v1`