Skip to content

Conversation

@nqd
Copy link
Collaborator

@nqd nqd commented Aug 4, 2025

Create a page manager trait so that we can experiment with different implementations, such as mmap or a buffer pool.

@cb-heimdall
Copy link
Collaborator

cb-heimdall commented Aug 4, 2025

🟡 Heimdall Review Status

Requirement Status More Info
Reviews 🟡 0/1
Denominator calculation
Show calculation
1 if user is bot 0
1 if user is external 0
2 if repo is sensitive 0
From .codeflow.yml 1
Additional review requirements
Show calculation
Max 0
0
From CODEOWNERS 0
Global minimum 0
Max 1
1
1 if commit is unverified 1
Sum 2

@nqd nqd force-pushed the page-manager-trait branch from c025340 to c406a82 Compare August 4, 2025 15:32
@nqd nqd marked this pull request as ready for review August 4, 2025 15:32
@nqd nqd requested review from BrianBland and and-cb August 4, 2025 15:40
BrianBland
BrianBland previously approved these changes Aug 5, 2025
#[derive(Debug)]
pub struct StorageEngine {
pub(crate) page_manager: PageManager,
pub(crate) page_manager: Box<dyn PageManagerTrait>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll probably want to use a generic type here, but this is fine for a first pass

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could use generic type, but that generic type will be bubbled to higher structs. I cannot find a good way to avoid <P: PageManagerTrait> at Database and at the struct that uses Database. Any suggestion for that?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically we could encapsulate P by using a PageManager enum type containing each of the possible page manager implementations. This may be the right thing to do in order to hide the PageManager concept entirely from clients.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't really need a trait for this, unless you want to be able to run multiple implementations in parallel. I would just use a Cargo feature

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@and-cb i just want to run one implementation at one time, and want an easy way to configure (benchmark program, reth). I am not sure with cargo flag, we have any better way than this?

#[cfg(feature = "mmap-backend")]
let db = Database::open(...)?;

#[cfg(feature = "bufferpool-backend")]  
let db = Database::open(...)?;

cargo test --features mmap-backend
cargo test --features bufferpool-backend

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BrianBland would the use of enum need a swithching like this for every common fn

    fn get(&self, page_id: PageId) -> Result<Page<'_>, PageError> {
        match self {
            Self::Mmap(manager) => manager.get(page_id),
            ...
        }
    }

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cargo test --features xyz part is correct; the #[cfg(feature = "xyz")] needs to be put above the PageManager implementations. Example:

// in src/page/manager.rs

#[cfg(feature = "mmap-backend")]
mod mmap;

#[cfg(feature = "bufferpool-backend")]
mod bufferpool;
// in src/page.rs

#[cfg(feature = "mmap-backend")]
pub use manager::mmap::PageManager;

#[cfg(feature = "bufferpool-backend")]
pub use manager::bufferpool::PageManager;

Now every code that does use crate::page::PageManager will get the one specified through --features

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think i will go with features from cargo. Thanks for suggestions

@cb-heimdall cb-heimdall dismissed BrianBland’s stale review August 5, 2025 01:38

Approved review 3086183780 from BrianBland is now dismissed due to new commit. Re-request for approval.

@nqd nqd requested a review from BrianBland August 5, 2025 02:40
@nqd nqd closed this Sep 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants