feat(conductor)!: implement v2 execution API #2006

ethanoroshiba · 2025-02-27T20:40:51Z

Summary

Updates Conductor to use v2 Execution API, including use of new "execution sessions".

Background

These changes are necessary to support migrating Forma to mainnet.

Migrating Forma to Astria's main net requires introducing stop and start logic into the Conductor. A migration using these changes could look like this:

Running against a fresh rollup node, Conductor starts and initiates a new execution session via astria.execution.v2.CreateExecutionSession, which contains rollup_start_block_number = 1, and rollup_end_block_number = 1000.
Conductor executes (syncs) the rollup node from rollup block numbers 1 to 1000 (inclusive)
Upon executing rollup number 1000, conductor resets its inner state and initiates a new execution session via astria.execution.v2.CreateExecutionSession, with new information from which it will continue executing.

For cases where rollup_end_block_number is unset (or equivalently set to 0), conductor will never restart as there is no stop condition.

For conductors running in firm-only or in soft-and-firm modes, only the firm-executed block triggers a restart (in soft-and-firm mode, upon soft-executing at the rollup stop number no blocks will be executed past it).

For conductors running in soft-only mode the soft-executed block triggers the restart.

If the rollup chain is halted, it should respond with tonic::Code::FailedPrecondition to CreateExecutionSession, which will cause the Conductor to shut down upon attempting a restart.

The aforementioned rollup changes to be made in astria-geth necessitate changing our current charts config for the following. "Forks" now describes a set of instructions for when and how to stop and restart the EVM at different heights, such that changes (such as network migration) can be made.

config:
        # < non astria-prefixed items... >
        AstriaOverrideGenesisExtraData:
        AstriaRollupName:
        AstriaForks: []
                # - < fork name >:
                #         height: ""
                #         halt: ""
                #         snapshotChecksum: ""
                #         extraDataOverride: ""
                #         feeCollector: ""
                #         EIP1559Params: {}
                #         sequencer:
                #                 chainId: ""
                #                 addressPrefix: ""
                #                 startHeight: ""
                #         celestia:
                #                 chainId: ""
                #                 startHeight: ""
                #                 searchHeightMaxLookAhead: ""
                #         bridgeAddresses: []
                #         astriaOracleCallerAddress: ""
                #         astriaOracleContractAddress: ""

Changes

In `astria-core`

Replaced v1 execution domain types with v2 execution domain types:
- ExecutionSession
- ExecutionSessionParameters
- ExecutedBlockMetadata
- CommitmentState (name unchanged, but fields are)
For more information on API changes, see feat(proto): create v2 execution API #1962

In `astria-auctioneer`

Updated execution types to v2

In `astria-conductor`

Changed sequencer-to-rollup-height calculation to start from rollup_start_block_number in ExecutionSessionParameters, so that Conductor doesn't start from rollup height 1 again after restarting.
Removed ASTRIA_CONDUCTOR_SEQUENCER_CHAIN_ID and ASTRIA_CONDUCTOR_CELESTIA_CHAIN_ID env vars from Conductor (chain ID checks will now be performed against the values retrieved from the genesis info)
Added restart logic for when the stop block number is reached: Conductor will restart at the stop height and initiate a new execution session
Removed multiplication of Celestia variance.

In `charts`

Added astriaForks to EVM rollup config, to reflect changes in astria-geth: https://github.com/astriaorg/astria-geth/pull/59/files.
Add just command for deploying the EVM rollup restart test, which functions the same as our regular smoke test with a restart at sequencer height 20.

Testing

Added miscellaneous unit tests to ensure heights are correctly calculated and whether to restart is successfully determined.
Added blackbox tests to ensure conductor correctly restarts in each mode, and continues to execute at the correct height after initiating a new execution session.
Added EVM rollup restart test which restarts EVM and Conductor at sequencer height 20 during the regular smoke test.

Changelogs

Changelogs updated

Breaking Changelist

Domain v1 execution types have been almost entirely replaced
Conductor config is not backwards compatible, i.e. old conductor will work with new config but not vice versa.
Conductor now required v2 execution API

Related Issues

closes #1994
closes #2003

commit ead3a9b Author: ethanoroshiba <ethan@astria.org> Date: Mon Feb 10 17:23:16 2025 -0600 make start height the first execution height instead of one before commit d8f2f3f Author: ethanoroshiba <ethan@astria.org> Date: Mon Feb 10 16:48:39 2025 -0600 break proto changes into v2 commit dd16dc3 Author: ethanoroshiba <ethan@astria.org> Date: Mon Feb 10 12:30:34 2025 -0600 Fix evm restart test values, some verbiage changes for clarity commit 260e24c Author: ethanoroshiba <ethan@astria.org> Date: Mon Feb 10 10:20:18 2025 -0600 fix geth genesis commit 3cee398 Author: ethanoroshiba <ethan@astria.org> Date: Mon Feb 10 10:19:20 2025 -0600 fix values commit 0bfd35f Author: ethanoroshiba <ethan@astria.org> Date: Mon Feb 10 09:14:45 2025 -0600 fix rollup name in evm restart test dev values commit 8ade517 Author: ethanoroshiba <ethan@astria.org> Date: Mon Feb 10 08:56:16 2025 -0600 give evm rollup restart test its own just command commit b2af17c Merge: 1e1668e 60eefa9 Author: Richard Janis Goldschmidt <github@aberrat.io> Date: Mon Feb 10 13:53:08 2025 +0100 Merge branch 'main' into superfluffy/forma-restart-logic commit 1e1668e Author: ethanoroshiba <ethan@astria.org> Date: Fri Feb 7 13:40:56 2025 -0600 remove configmap values commit 782ee12 Author: ethanoroshiba <ethan@astria.org> Date: Fri Feb 7 13:36:55 2025 -0600 fix evm restart test heights, add env vars back to non dev configmap commit f5051d5 Author: Richard Janis Goldschmidt <github@aberrat.io> Date: Tue Feb 4 20:05:27 2025 +0100 remove grpc mock changes after rebasing on main commit 98d04a8 Author: Richard Janis Goldschmidt <github@aberrat.io> Date: Tue Feb 4 18:16:02 2025 +0100 fix proto formatting commit 373aaa0 Author: Richard Janis Goldschmidt <github@aberrat.io> Date: Tue Feb 4 18:09:46 2025 +0100 rename halt_at_stop_height -> halt_at_rollup_stop_height commit 6b9b54a Author: ethanoroshiba <ethan@astria.org> Date: Tue Feb 4 10:30:37 2025 -0600 bump charts, format core changelog commit 50010e9 Author: Richard Janis Goldschmidt <github@aberrat.io> Date: Tue Feb 4 14:43:14 2025 +0100 sequencer_end_height -> sequencer_stop_height commit 0be83ab Author: Richard Janis Goldschmidt <github@aberrat.io> Date: Tue Feb 4 14:36:50 2025 +0100 updated conductor, core changelogs commit faa42af Author: Richard Janis Goldschmidt <github@aberrat.io> Date: Tue Feb 4 14:32:09 2025 +0100 remove tendermint ID from public API of execution genesis commit 31caab9 Author: Richard Janis Goldschmidt <github@aberrat.io> Date: Tue Feb 4 14:19:02 2025 +0100 rename last_height -> stop_height commit 7b5b5fc Author: Richard Janis Goldschmidt <github@aberrat.io> Date: Tue Feb 4 14:10:19 2025 +0100 bump charts commit 4eb7806 Author: Richard Janis Goldschmidt <github@aberrat.io> Date: Tue Feb 4 13:56:57 2025 +0100 remove confusing test helper commit c586b33 Author: Richard Janis Goldschmidt <github@aberrat.io> Date: Tue Feb 4 13:53:18 2025 +0100 remove conductor_ prefix from tests, remove explicit sleep commit 42ae864 Author: ethanoroshiba <ethan@astria.org> Date: Thu Jan 30 11:54:10 2025 -0600 bring back executor tests, make test utils commit 66315e7 Author: ethanoroshiba <ethan@astria.org> Date: Thu Jan 30 11:36:14 2025 -0600 base stop height on rollup instead of sequencer commit 4ea3300 Author: ethanoroshiba <ethan@astria.org> Date: Tue Jan 28 12:33:39 2025 -0600 fix typos commit c720931 Author: ethanoroshiba <ethan@astria.org> Date: Tue Jan 28 12:26:14 2025 -0600 update changelog, example config, test soft-first and firm-first scenarios separately commit add58bc Author: ethanoroshiba <ethan@astria.org> Date: Mon Jan 27 12:29:55 2025 -0600 revert range changes commit d12ed9a Author: ethanoroshiba <ethan@astria.org> Date: Mon Jan 27 12:21:22 2025 -0600 add necessary charts changes and smoke test, change stop height to be inclusive commit a45206c Author: Richard Janis Goldschmidt <github@aberrat.io> Date: Wed Jan 22 06:14:49 2025 +0100 feat(conductor): restart or stop at set height Co-authored-by: Ethan Oroshiba <ethan@astria.org>

SuperFluffy

Thank you so much for slogging this out. Please address the last comments.

crates/astria-conductor/src/executor/client.rs

crates/astria-conductor/src/executor/mod.rs

crates/astria-conductor/src/state.rs

crates/astria-conductor/src/executor/mod.rs

Co-authored-by: Richard Janis Goldschmidt <github@aberrat.io>

## Description  Updates the Execution API docs to reflect latest updates to v2: astriaorg/astria#2006 and astriaorg/astria#1962 ## Motivation and Context   The v2 Execution API starkly contrasts to v1, so changing the documentation seemed appropriate. ## Screenshots (if appropriate):  ## Types of changes  - [x] Edits to existing documentation - [ ] Changing documentation structure (relocating existing files, ensure redirects exist) - [ ] Stylistic changes (provide screenshots above)

ethanoroshiba added 2 commits February 26, 2025 12:57

implement v2 execution api

959e08f

ethanoroshiba added enhancement New feature or request conductor pertaining to the astria-conductor crate evm pertains to EVM docker-build used to trigger docker builds on PRs override-freeze labels Feb 27, 2025

github-actions bot added documentation Improvements or additions to documentation ci issues that are related to ci and github workflows proto pertaining to the Astria Protobuf spec cd labels Feb 27, 2025

ethanoroshiba temporarily deployed to BUF February 27, 2025 20:41 — with GitHub Actions Inactive

update charts, changelog, and spec

5819f9b

ethanoroshiba temporarily deployed to BUF February 27, 2025 20:51 — with GitHub Actions Inactive

bump chart versions

93d0ebe

ethanoroshiba changed the title ~~feat(conductor): implement v2 execution API~~ feat(conductor)!: implement v2 execution API Feb 27, 2025

ethanoroshiba temporarily deployed to BUF February 27, 2025 21:08 — with GitHub Actions Inactive

revert execution api spec changes

8dfc893

ethanoroshiba temporarily deployed to BUF February 27, 2025 21:12 — with GitHub Actions Inactive

fix tests and conductor logic

1339997

ethanoroshiba temporarily deployed to BUF March 4, 2025 20:00 — with GitHub Actions Inactive

ethanoroshiba requested review from SuperFluffy, joroshiba and mycodecrafting March 4, 2025 20:17

ethanoroshiba marked this pull request as ready for review March 4, 2025 20:17

ethanoroshiba requested review from a team, Fraser999 and bharath-123 as code owners March 4, 2025 20:17

SuperFluffy approved these changes Apr 8, 2025

View reviewed changes

ethanoroshiba added 2 commits April 9, 2025 12:25

requested changes

50b3a45

Merge branch 'main' into eoroshiba/conductor-execution-sessions

bde71c2

ethanoroshiba temporarily deployed to BUF April 9, 2025 17:28 — with GitHub Actions Inactive

fix changes from merge

0621953

ethanoroshiba temporarily deployed to BUF April 9, 2025 17:34 — with GitHub Actions Inactive

SuperFluffy reviewed Apr 9, 2025

View reviewed changes

requested changes

54d805f

ethanoroshiba temporarily deployed to BUF April 9, 2025 20:12 — with GitHub Actions Inactive

SuperFluffy reviewed Apr 9, 2025

View reviewed changes

crates/astria-conductor/src/executor/mod.rs Outdated Show resolved Hide resolved

Suggested change

b9d012d

Co-authored-by: Richard Janis Goldschmidt <github@aberrat.io>

ethanoroshiba temporarily deployed to BUF April 9, 2025 20:26 — with GitHub Actions Inactive

rustfmt

c8e630f

ethanoroshiba temporarily deployed to BUF April 9, 2025 20:27 — with GitHub Actions Inactive

ethanoroshiba added 2 commits April 10, 2025 16:09

Merge branch 'main' into eoroshiba/conductor-execution-sessions

5249bfa

fix genesis and smoke test

2e49baf

ethanoroshiba temporarily deployed to BUF April 10, 2025 21:18 — with GitHub Actions Inactive

Merge branch 'main' into eoroshiba/conductor-execution-sessions

8b78c25

ethanoroshiba temporarily deployed to BUF April 21, 2025 13:54 — with GitHub Actions Inactive

fix changes from merge

6dd481c

ethanoroshiba temporarily deployed to BUF April 21, 2025 14:01 — with GitHub Actions Inactive

aajimal approved these changes Apr 21, 2025

View reviewed changes

update geth tag to latest

9467193

ethanoroshiba temporarily deployed to BUF April 21, 2025 17:56 — with GitHub Actions Inactive

ethanoroshiba added this pull request to the merge queue Apr 21, 2025

Merged via the queue into main with commit 007217c Apr 21, 2025
50 checks passed

ethanoroshiba deleted the eoroshiba/conductor-execution-sessions branch April 21, 2025 18:20

ethanoroshiba mentioned this pull request Apr 23, 2025

chore: update execution API docs astriaorg/docs#124

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(conductor)!: implement v2 execution API #2006

feat(conductor)!: implement v2 execution API #2006

Uh oh!

ethanoroshiba commented Feb 27, 2025 •

edited

Loading

Uh oh!

SuperFluffy left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

feat(conductor)!: implement v2 execution API #2006

feat(conductor)!: implement v2 execution API #2006

Uh oh!

Conversation

ethanoroshiba commented Feb 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Background

Changes

In astria-core

In astria-auctioneer

In astria-conductor

In charts

Testing

Changelogs

Breaking Changelist

Related Issues

Uh oh!

SuperFluffy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ethanoroshiba commented Feb 27, 2025 •

edited

Loading

In `astria-core`

In `astria-auctioneer`

In `astria-conductor`

In `charts`