aya-ebpf: Add BTF map definitions for hash maps #1367
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for aya-rs-docs ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
f93c56a to
131c0b3
Compare
There was a problem hiding this comment.
Pull Request Overview
Introduces BTF map definitions for hash maps, enabling developers to define hash map structures that are embedded in the .maps section with proper metadata. This adds support for four types of hash maps: HashMap, LruHashMap, PerCpuHashMap, and LruPerCpuHashMap with corresponding BTF definitions.
Key changes include:
- Added new BTF hash map types and definitions
- Enhanced documentation for map safety concerns
- Added comprehensive integration tests for all hash map types
Reviewed Changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| xtask/public-api/aya-ebpf.txt | Updates public API surface to include new hash map types and methods |
| test/integration-test/src/tests/hash_map.rs | Comprehensive test suite for all hash map variants with both BTF and legacy definitions |
| test/integration-test/src/tests.rs | Adds hash_map module to test suite |
| test/integration-test/src/lib.rs | Registers hash_map binary for testing |
| test/integration-test/Cargo.toml | Adds nix dependency for CPU scheduling in tests |
| test/integration-ebpf/src/hash_map.rs | eBPF implementation with BTF and legacy map definitions for testing |
| test/integration-ebpf/Cargo.toml | Adds hash_map binary target |
| test/integration-common/src/lib.rs | Defines constants for hash map testing |
| ebpf/aya-ebpf/src/maps/map_safety.md | Detailed documentation on map safety considerations |
| ebpf/aya-ebpf/src/maps/hash_map.rs | Updates existing hash map documentation to reference safety documentation |
| ebpf/aya-ebpf/src/btf_maps/mod.rs | Exports new hash map types from btf_maps module |
| ebpf/aya-ebpf/src/btf_maps/hash_map.rs | Implementation of BTF hash map types with proper struct definitions |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Comment on lines
128
to
134
| pub struct PerCpuHashMap<K, V, const M: usize, const F: usize>( | ||
| UnsafeCell<PerCpuHashMapDef<K, V, M, F>>, | ||
| ); | ||
|
|
||
| unsafe impl<K: Sync, V: Sync, const M: usize, const F: usize> Sync for PerCpuHashMap<K, V, M, F> {} | ||
|
|
||
| impl<K, V, const M: usize, const F: usize> PerCpuHashMap<K, V, M, F> { |
There was a problem hiding this comment.
[nitpick] Inconsistent default values for the F parameter. HashMap and LruHashMap have const F: usize = 0 while PerCpuHashMap doesn't provide a default. This inconsistency could confuse users.
Suggested change
| pub struct PerCpuHashMap<K, V, const M: usize, const F: usize>( | |
| UnsafeCell<PerCpuHashMapDef<K, V, M, F>>, | |
| ); | |
| unsafe impl<K: Sync, V: Sync, const M: usize, const F: usize> Sync for PerCpuHashMap<K, V, M, F> {} | |
| impl<K, V, const M: usize, const F: usize> PerCpuHashMap<K, V, M, F> { | |
| pub struct PerCpuHashMap<K, V, const M: usize, const F: usize = 0>( | |
| UnsafeCell<PerCpuHashMapDef<K, V, M, F>>, | |
| ); | |
| unsafe impl<K: Sync, V: Sync, const M: usize, const F: usize = 0> Sync for PerCpuHashMap<K, V, M, F> {} | |
| impl<K, V, const M: usize, const F: usize = 0> PerCpuHashMap<K, V, M, F> { |
Member
Author
There was a problem hiding this comment.
That suggestion is incorrect - you can't provide defaults for const parameters (= 0) in impl blocks:
cargo:warning=error: defaults for const parameters are only allowed in `struct`, `enum`, `type`, or `trait` definitions
cargo:warning= --> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:132:47
cargo:warning= |
cargo:warning=132 | unsafe impl<K: Sync, V: Sync, const M: usize, const F: usize = 0> Sync
cargo:warning= |
And that makes sense - defaults are used when the struct with generics is initialized. impl blocks shouldn't care about them.
131c0b3 to
5ca275a
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
tamird
requested changes
Oct 14, 2025
Member
tamird
left a comment
There was a problem hiding this comment.
@tamird reviewed 2 of 2 files at r1, 10 of 10 files at r3, 9 of 10 files at r4, all commit messages.
Reviewable status: 11 of 12 files reviewed, 8 unresolved discussions (waiting on @vadorovsky)
ebpf/aya-ebpf/src/maps/map_safety.md line 3 at r1 (raw file):
# Safety The pointer returned by a BPF map lookup is only stable until a concurrent
"concurrent"?
ebpf/aya-ebpf/src/btf_maps/hash_map.rs line 1 at r4 (raw file):
use core::{borrow::Borrow, cell::UnsafeCell};
maybe warn missing docs in the whole btf_maps module?
ebpf/aya-ebpf/src/btf_maps/hash_map.rs line 20 at r4 (raw file):
unsafe impl<K: Sync, V: Sync, const M: usize, const F: usize> Sync for HashMap<K, V, M, F> {} impl<K, V, const M: usize, const F: usize> HashMap<K, V, M, F> {
are these 4 maps all basically identical? can we use a macro or otherwise reduce all the duplication?
ebpf/aya-ebpf/src/maps/hash_map.rs line 72 at r1 (raw file):
} /// Inserts a key-value pair into the map.
Can we warn on missing docs now to prevent backsliding?
test/integration-test/src/tests/hash_map.rs line 12 at r4 (raw file):
sched::{CpuSet, sched_setaffinity}, unistd::Pid, };
can you remove this import? this is the kind of thing where i really wanna grep for nix:: and see the usage
Code quote:
use nix::{
sched::{CpuSet, sched_setaffinity},
unistd::Pid,
};
Member
|
@codex review |
|
Codex Review: Didn't find any major issues. Chef's kiss. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. |
Member
Author
|
Looks like CI is not happy about the LRU map tests. 🙄 Will debug tomorrow, it's zzz time for me. They all work for me locally, but I noticed a weird thing in the per-CPU LRU map: the |
5ca275a to
4b4ce8c
Compare
vadorovsky
commented
Oct 16, 2025
Member
Author
vadorovsky
left a comment
There was a problem hiding this comment.
Reviewable status: 11 of 12 files reviewed, 8 unresolved discussions (waiting on @tamird)
ebpf/aya-ebpf/src/btf_maps/hash_map.rs line 1 at r4 (raw file):
Previously, tamird (Tamir Duberstein) wrote…
maybe warn missing docs in the whole btf_maps module?
Great idea, but I would do it separately. There are a lot of missing docs even outside of hash maps:
error: missing documentation for a module
--> ebpf/aya-ebpf/src/lib.rs:30:1
|
30 | pub mod btf_maps;
| ^^^^^^^^^^^^^^^^
|
note: the lint level is defined here
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:1:9
|
1 | #![deny(missing_docs)]
| ^^^^^^^^^^^^
error: missing documentation for a module
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:5:1
|
5 | pub mod array;
| ^^^^^^^^^^^^^
error: missing documentation for a module
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:6:1
|
6 | pub mod hash_map;
| ^^^^^^^^^^^^^^^^
error: missing documentation for a module
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:7:1
|
7 | pub mod sk_storage;
| ^^^^^^^^^^^^^^^^^^
error: missing documentation for a macro
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:25:1
|
25 | macro_rules! btf_map_def {
| ^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for a struct
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:31:9
|
31 | / pub struct $name<K, V, const M: usize, const F: usize = 0> {
32 | | r#type: *const [i32; $t as usize],
33 | | key: *const K,
34 | | value: *const V,
... |
39 | | _anon: $crate::btf_maps::AyaBtfMapMarker,
40 | | }
| |_________^
|
::: ebpf/aya-ebpf/src/btf_maps/array.rs:5:1
|
5 | btf_map_def!(ArrayDef, BPF_MAP_TYPE_ARRAY);
| ------------------------------------------ in this macro invocation
|
= note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info)
error: missing documentation for an associated function
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:47:13
|
47 | pub const fn new() -> $name<K, V, M, F> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
::: ebpf/aya-ebpf/src/btf_maps/array.rs:5:1
|
5 | btf_map_def!(ArrayDef, BPF_MAP_TYPE_ARRAY);
| ------------------------------------------ in this macro invocation
|
= note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info)
error: missing documentation for a struct
--> ebpf/aya-ebpf/src/btf_maps/array.rs:8:1
|
8 | pub struct Array<T, const M: usize, const F: usize = 0>(UnsafeCell<ArrayDef<u32, T, M, F>>);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for a method
--> ebpf/aya-ebpf/src/btf_maps/array.rs:33:5
|
33 | pub fn get(&self, index: u32) -> Option<&T> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for a method
--> ebpf/aya-ebpf/src/btf_maps/array.rs:38:5
|
38 | pub fn get_ptr(&self, index: u32) -> Option<*const T> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for a method
--> ebpf/aya-ebpf/src/btf_maps/array.rs:43:5
|
43 | pub fn get_ptr_mut(&self, index: u32) -> Option<*mut T> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for a struct
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:31:9
|
31 | / pub struct $name<K, V, const M: usize, const F: usize = 0> {
32 | | r#type: *const [i32; $t as usize],
33 | | key: *const K,
34 | | value: *const V,
... |
39 | | _anon: $crate::btf_maps::AyaBtfMapMarker,
40 | | }
| |_________^
|
::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:13:1
|
13 | btf_map_def!(HashMapDef, BPF_MAP_TYPE_HASH);
| ------------------------------------------- in this macro invocation
|
= note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info)
error: missing documentation for an associated function
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:47:13
|
47 | pub const fn new() -> $name<K, V, M, F> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:13:1
|
13 | btf_map_def!(HashMapDef, BPF_MAP_TYPE_HASH);
| ------------------------------------------- in this macro invocation
|
= note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info)
error: missing documentation for a struct
--> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:16:1
|
16 | pub struct HashMap<K, V, const M: usize, const F: usize = 0>(UnsafeCell<HashMapDef<K, V, M, F>>);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for an associated function
--> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:25:5
|
25 | pub const fn new() -> Self {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for a struct
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:31:9
|
31 | / pub struct $name<K, V, const M: usize, const F: usize = 0> {
32 | | r#type: *const [i32; $t as usize],
33 | | key: *const K,
34 | | value: *const V,
... |
39 | | _anon: $crate::btf_maps::AyaBtfMapMarker,
40 | | }
| |_________^
|
::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:68:1
|
68 | btf_map_def!(LruHashMapDef, BPF_MAP_TYPE_LRU_HASH);
| -------------------------------------------------- in this macro invocation
|
= note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info)
error: missing documentation for an associated function
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:47:13
|
47 | pub const fn new() -> $name<K, V, M, F> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:68:1
|
68 | btf_map_def!(LruHashMapDef, BPF_MAP_TYPE_LRU_HASH);
| -------------------------------------------------- in this macro invocation
|
= note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info)
error: missing documentation for a struct
--> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:71:1
|
71 | pub struct LruHashMap<K, V, const M: usize, const F: usize = 0>(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for an associated function
--> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:82:5
|
82 | pub const fn new() -> Self {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for a struct
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:31:9
|
31 | / pub struct $name<K, V, const M: usize, const F: usize = 0> {
32 | | r#type: *const [i32; $t as usize],
33 | | key: *const K,
34 | | value: *const V,
... |
39 | | _anon: $crate::btf_maps::AyaBtfMapMarker,
40 | | }
| |_________^
|
::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:125:1
|
125 | btf_map_def!(PerCpuHashMapDef, BPF_MAP_TYPE_PERCPU_HASH);
| -------------------------------------------------------- in this macro invocation
|
= note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info)
error: missing documentation for an associated function
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:47:13
|
47 | pub const fn new() -> $name<K, V, M, F> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:125:1
|
125 | btf_map_def!(PerCpuHashMapDef, BPF_MAP_TYPE_PERCPU_HASH);
| -------------------------------------------------------- in this macro invocation
|
= note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info)
error: missing documentation for a struct
--> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:128:1
|
128 | pub struct PerCpuHashMap<K, V, const M: usize, const F: usize = 0>(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for an associated function
--> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:139:5
|
139 | pub const fn new() -> Self {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for a struct
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:31:9
|
31 | / pub struct $name<K, V, const M: usize, const F: usize = 0> {
32 | | r#type: *const [i32; $t as usize],
33 | | key: *const K,
34 | | value: *const V,
... |
39 | | _anon: $crate::btf_maps::AyaBtfMapMarker,
40 | | }
| |_________^
|
::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:182:1
|
182 | btf_map_def!(LruPerCpuHashMapDef, BPF_MAP_TYPE_LRU_PERCPU_HASH);
| --------------------------------------------------------------- in this macro invocation
|
= note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info)
error: missing documentation for an associated function
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:47:13
|
47 | pub const fn new() -> $name<K, V, M, F> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:182:1
|
182 | btf_map_def!(LruPerCpuHashMapDef, BPF_MAP_TYPE_LRU_PERCPU_HASH);
| --------------------------------------------------------------- in this macro invocation
|
= note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info)
error: missing documentation for a struct
--> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:185:1
|
185 | pub struct LruPerCpuHashMap<K, V, const M: usize, const F: usize = 0>(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for an associated function
--> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:199:5
|
199 | pub const fn new() -> Self {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for a struct
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:31:9
|
31 | / pub struct $name<K, V, const M: usize, const F: usize = 0> {
32 | | r#type: *const [i32; $t as usize],
33 | | key: *const K,
34 | | value: *const V,
... |
39 | | _anon: $crate::btf_maps::AyaBtfMapMarker,
40 | | }
| |_________^
|
::: ebpf/aya-ebpf/src/btf_maps/sk_storage.rs:14:1
|
14 | btf_map_def!(SkStorageDef, BPF_MAP_TYPE_SK_STORAGE);
| --------------------------------------------------- in this macro invocation
|
= note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info)
error: missing documentation for an associated function
--> ebpf/aya-ebpf/src/btf_maps/mod.rs:47:13
|
47 | pub const fn new() -> $name<K, V, M, F> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
::: ebpf/aya-ebpf/src/btf_maps/sk_storage.rs:14:1
|
14 | btf_map_def!(SkStorageDef, BPF_MAP_TYPE_SK_STORAGE);
| --------------------------------------------------- in this macro invocation
|
= note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info)
error: missing documentation for a struct
--> ebpf/aya-ebpf/src/btf_maps/sk_storage.rs:20:1
|
20 | pub struct SkStorage<T>(UnsafeCell<SkStorageDef<i32, T, 0, { BPF_F_NO_PREALLOC as usize }>>);
| ^^^^^^^^^^^^^^^^^^^^^^^
error: missing documentation for an associated function
--> ebpf/aya-ebpf/src/btf_maps/sk_storage.rs:29:5
|
29 | pub const fn new() -> Self {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
error: could not compile `aya-ebpf` (lib) due to 31 previous errors
ebpf/aya-ebpf/src/btf_maps/hash_map.rs line 20 at r4 (raw file):
Previously, tamird (Tamir Duberstein) wrote…
are these 4 maps all basically identical? can we use a macro or otherwise reduce all the duplication?
Done, good call.
ebpf/aya-ebpf/src/maps/hash_map.rs line 72 at r1 (raw file):
Previously, tamird (Tamir Duberstein) wrote…
Can we warn on missing docs now to prevent backsliding?
Same as above - it would require some work unrelated to hash maps. Happy to do that in a separate PR.
ebpf/aya-ebpf/src/maps/map_safety.md line 3 at r1 (raw file):
Previously, tamird (Tamir Duberstein) wrote…
"concurrent"?
Yes, concurrent. Concurrent updates and deletes from different program calls (or different programs) are a problem. That sentence pasted from your suggestion on Discord, BTW. 🙂 But feel free to propose a better wording.
test/integration-test/src/tests/hash_map.rs line 12 at r4 (raw file):
Previously, tamird (Tamir Duberstein) wrote…
can you remove this import? this is the kind of thing where i really wanna grep for nix:: and see the usage
Done.
d92502b to
cd76d8a
Compare
tamird
requested changes
Oct 16, 2025
Member
tamird
left a comment
There was a problem hiding this comment.
That means - if you have a map with max limit 10 and you insert 5 elements from 2 different threads (each pinned on different CPU), you already hit the limit and adding more elements on any core triggers eviction. That's the reason why I ended up keeping just one pinned thread for now.
I think that's because you are attaching N copies of the BPF program. See my comments in the test.
@tamird reviewed 5 of 19 files at r5, 18 of 18 files at r7, 14 of 14 files at r8, all commit messages.
Reviewable status: all files reviewed, 12 unresolved discussions (waiting on @vadorovsky)
ebpf/aya-ebpf/src/btf_maps/hash_map.rs line 1 at r4 (raw file):
Previously, vadorovsky (Michal R) wrote…
Great idea, but I would do it separately. There are a lot of missing docs even outside of hash maps:
error: missing documentation for a module --> ebpf/aya-ebpf/src/lib.rs:30:1 | 30 | pub mod btf_maps; | ^^^^^^^^^^^^^^^^ | note: the lint level is defined here --> ebpf/aya-ebpf/src/btf_maps/mod.rs:1:9 | 1 | #![deny(missing_docs)] | ^^^^^^^^^^^^ error: missing documentation for a module --> ebpf/aya-ebpf/src/btf_maps/mod.rs:5:1 | 5 | pub mod array; | ^^^^^^^^^^^^^ error: missing documentation for a module --> ebpf/aya-ebpf/src/btf_maps/mod.rs:6:1 | 6 | pub mod hash_map; | ^^^^^^^^^^^^^^^^ error: missing documentation for a module --> ebpf/aya-ebpf/src/btf_maps/mod.rs:7:1 | 7 | pub mod sk_storage; | ^^^^^^^^^^^^^^^^^^ error: missing documentation for a macro --> ebpf/aya-ebpf/src/btf_maps/mod.rs:25:1 | 25 | macro_rules! btf_map_def { | ^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for a struct --> ebpf/aya-ebpf/src/btf_maps/mod.rs:31:9 | 31 | / pub struct $name<K, V, const M: usize, const F: usize = 0> { 32 | | r#type: *const [i32; $t as usize], 33 | | key: *const K, 34 | | value: *const V, ... | 39 | | _anon: $crate::btf_maps::AyaBtfMapMarker, 40 | | } | |_________^ | ::: ebpf/aya-ebpf/src/btf_maps/array.rs:5:1 | 5 | btf_map_def!(ArrayDef, BPF_MAP_TYPE_ARRAY); | ------------------------------------------ in this macro invocation | = note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info) error: missing documentation for an associated function --> ebpf/aya-ebpf/src/btf_maps/mod.rs:47:13 | 47 | pub const fn new() -> $name<K, V, M, F> { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ::: ebpf/aya-ebpf/src/btf_maps/array.rs:5:1 | 5 | btf_map_def!(ArrayDef, BPF_MAP_TYPE_ARRAY); | ------------------------------------------ in this macro invocation | = note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info) error: missing documentation for a struct --> ebpf/aya-ebpf/src/btf_maps/array.rs:8:1 | 8 | pub struct Array<T, const M: usize, const F: usize = 0>(UnsafeCell<ArrayDef<u32, T, M, F>>); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for a method --> ebpf/aya-ebpf/src/btf_maps/array.rs:33:5 | 33 | pub fn get(&self, index: u32) -> Option<&T> { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for a method --> ebpf/aya-ebpf/src/btf_maps/array.rs:38:5 | 38 | pub fn get_ptr(&self, index: u32) -> Option<*const T> { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for a method --> ebpf/aya-ebpf/src/btf_maps/array.rs:43:5 | 43 | pub fn get_ptr_mut(&self, index: u32) -> Option<*mut T> { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for a struct --> ebpf/aya-ebpf/src/btf_maps/mod.rs:31:9 | 31 | / pub struct $name<K, V, const M: usize, const F: usize = 0> { 32 | | r#type: *const [i32; $t as usize], 33 | | key: *const K, 34 | | value: *const V, ... | 39 | | _anon: $crate::btf_maps::AyaBtfMapMarker, 40 | | } | |_________^ | ::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:13:1 | 13 | btf_map_def!(HashMapDef, BPF_MAP_TYPE_HASH); | ------------------------------------------- in this macro invocation | = note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info) error: missing documentation for an associated function --> ebpf/aya-ebpf/src/btf_maps/mod.rs:47:13 | 47 | pub const fn new() -> $name<K, V, M, F> { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:13:1 | 13 | btf_map_def!(HashMapDef, BPF_MAP_TYPE_HASH); | ------------------------------------------- in this macro invocation | = note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info) error: missing documentation for a struct --> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:16:1 | 16 | pub struct HashMap<K, V, const M: usize, const F: usize = 0>(UnsafeCell<HashMapDef<K, V, M, F>>); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for an associated function --> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:25:5 | 25 | pub const fn new() -> Self { | ^^^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for a struct --> ebpf/aya-ebpf/src/btf_maps/mod.rs:31:9 | 31 | / pub struct $name<K, V, const M: usize, const F: usize = 0> { 32 | | r#type: *const [i32; $t as usize], 33 | | key: *const K, 34 | | value: *const V, ... | 39 | | _anon: $crate::btf_maps::AyaBtfMapMarker, 40 | | } | |_________^ | ::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:68:1 | 68 | btf_map_def!(LruHashMapDef, BPF_MAP_TYPE_LRU_HASH); | -------------------------------------------------- in this macro invocation | = note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info) error: missing documentation for an associated function --> ebpf/aya-ebpf/src/btf_maps/mod.rs:47:13 | 47 | pub const fn new() -> $name<K, V, M, F> { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:68:1 | 68 | btf_map_def!(LruHashMapDef, BPF_MAP_TYPE_LRU_HASH); | -------------------------------------------------- in this macro invocation | = note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info) error: missing documentation for a struct --> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:71:1 | 71 | pub struct LruHashMap<K, V, const M: usize, const F: usize = 0>( | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for an associated function --> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:82:5 | 82 | pub const fn new() -> Self { | ^^^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for a struct --> ebpf/aya-ebpf/src/btf_maps/mod.rs:31:9 | 31 | / pub struct $name<K, V, const M: usize, const F: usize = 0> { 32 | | r#type: *const [i32; $t as usize], 33 | | key: *const K, 34 | | value: *const V, ... | 39 | | _anon: $crate::btf_maps::AyaBtfMapMarker, 40 | | } | |_________^ | ::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:125:1 | 125 | btf_map_def!(PerCpuHashMapDef, BPF_MAP_TYPE_PERCPU_HASH); | -------------------------------------------------------- in this macro invocation | = note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info) error: missing documentation for an associated function --> ebpf/aya-ebpf/src/btf_maps/mod.rs:47:13 | 47 | pub const fn new() -> $name<K, V, M, F> { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:125:1 | 125 | btf_map_def!(PerCpuHashMapDef, BPF_MAP_TYPE_PERCPU_HASH); | -------------------------------------------------------- in this macro invocation | = note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info) error: missing documentation for a struct --> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:128:1 | 128 | pub struct PerCpuHashMap<K, V, const M: usize, const F: usize = 0>( | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for an associated function --> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:139:5 | 139 | pub const fn new() -> Self { | ^^^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for a struct --> ebpf/aya-ebpf/src/btf_maps/mod.rs:31:9 | 31 | / pub struct $name<K, V, const M: usize, const F: usize = 0> { 32 | | r#type: *const [i32; $t as usize], 33 | | key: *const K, 34 | | value: *const V, ... | 39 | | _anon: $crate::btf_maps::AyaBtfMapMarker, 40 | | } | |_________^ | ::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:182:1 | 182 | btf_map_def!(LruPerCpuHashMapDef, BPF_MAP_TYPE_LRU_PERCPU_HASH); | --------------------------------------------------------------- in this macro invocation | = note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info) error: missing documentation for an associated function --> ebpf/aya-ebpf/src/btf_maps/mod.rs:47:13 | 47 | pub const fn new() -> $name<K, V, M, F> { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ::: ebpf/aya-ebpf/src/btf_maps/hash_map.rs:182:1 | 182 | btf_map_def!(LruPerCpuHashMapDef, BPF_MAP_TYPE_LRU_PERCPU_HASH); | --------------------------------------------------------------- in this macro invocation | = note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info) error: missing documentation for a struct --> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:185:1 | 185 | pub struct LruPerCpuHashMap<K, V, const M: usize, const F: usize = 0>( | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for an associated function --> ebpf/aya-ebpf/src/btf_maps/hash_map.rs:199:5 | 199 | pub const fn new() -> Self { | ^^^^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for a struct --> ebpf/aya-ebpf/src/btf_maps/mod.rs:31:9 | 31 | / pub struct $name<K, V, const M: usize, const F: usize = 0> { 32 | | r#type: *const [i32; $t as usize], 33 | | key: *const K, 34 | | value: *const V, ... | 39 | | _anon: $crate::btf_maps::AyaBtfMapMarker, 40 | | } | |_________^ | ::: ebpf/aya-ebpf/src/btf_maps/sk_storage.rs:14:1 | 14 | btf_map_def!(SkStorageDef, BPF_MAP_TYPE_SK_STORAGE); | --------------------------------------------------- in this macro invocation | = note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info) error: missing documentation for an associated function --> ebpf/aya-ebpf/src/btf_maps/mod.rs:47:13 | 47 | pub const fn new() -> $name<K, V, M, F> { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ::: ebpf/aya-ebpf/src/btf_maps/sk_storage.rs:14:1 | 14 | btf_map_def!(SkStorageDef, BPF_MAP_TYPE_SK_STORAGE); | --------------------------------------------------- in this macro invocation | = note: this error originates in the macro `btf_map_def` (in Nightly builds, run with -Z macro-backtrace for more info) error: missing documentation for a struct --> ebpf/aya-ebpf/src/btf_maps/sk_storage.rs:20:1 | 20 | pub struct SkStorage<T>(UnsafeCell<SkStorageDef<i32, T, 0, { BPF_F_NO_PREALLOC as usize }>>); | ^^^^^^^^^^^^^^^^^^^^^^^ error: missing documentation for an associated function --> ebpf/aya-ebpf/src/btf_maps/sk_storage.rs:29:5 | 29 | pub const fn new() -> Self { | ^^^^^^^^^^^^^^^^^^^^^^^^^^ error: could not compile `aya-ebpf` (lib) due to 31 previous errors
could do it just in this file though
ebpf/aya-ebpf/src/maps/hash_map.rs line 72 at r1 (raw file):
Previously, vadorovsky (Michal R) wrote…
Same as above - it would require some work unrelated to hash maps. Happy to do that in a separate PR.
You could probably do it for just this module, though.
ebpf/aya-ebpf/src/maps/map_safety.md line 3 at r1 (raw file):
Previously, vadorovsky (Michal R) wrote…
Yes, concurrent. Concurrent updates and deletes from different program calls (or different programs) are a problem. That sentence pasted from your suggestion on Discord, BTW. 🙂 But feel free to propose a better wording.
I think the word concurrent is superfluous here. The pointer is only valid until an update or delete, period. No?
xtask/public-api/aya-ebpf.txt line 411 at r7 (raw file):
pub fn aya_ebpf::maps::hash_map::PerCpuHashMap<K, V>::remove(&self, key: impl core::borrow::Borrow<K>) -> core::result::Result<(), aya_ebpf_cty::od::c_long> pub const fn aya_ebpf::maps::hash_map::PerCpuHashMap<K, V>::with_max_entries(max_entries: u32, flags: u32) -> Self impl<K: core::marker::Sync, V: core::marker::Sync> core::marker::Sync for aya_ebpf::maps::hash_map::PerCpuHashMap<K, V>
PerCpuHashMap should probably not be send + sync .. right?
ebpf/aya-ebpf/src/btf_maps/hash_map.rs line 27 at r8 (raw file):
pub struct $name<K, V, const M: usize, const F: usize = 0>(UnsafeCell<$def<K, V, M, F>>); unsafe impl<K: Sync, V: Sync, const M: usize, const F: usize> Sync for $name<K, V, M, F> {}
same here, i think per-cpu maps should probably not be sync?
test/integration-test/src/tests/hash_map.rs line 73 at r8 (raw file):
#[test_log::test] fn test_hash_map() {
all these separate tests are admirable but i think result in boilerplate that doesn't add value. just do it all in one pass IMO.
test/integration-test/src/tests/hash_map.rs line 171 at r8 (raw file):
("PER_CPU_HASH_MAP_LEGACY", "per_cpu_hash_map_insert_legacy"), ] { let hash_map = load_program_with_per_cpu_map(&mut ebpf, hash_map_name, insert_prog_name);
does it make sense to do this in a loop? i'm not sure what is happening, but i think you're getting N copies of the program all attached.
test/integration-test/src/tests/hash_map.rs line 189 at r8 (raw file):
let key = i.pow(2); let values = hash_map.get(&key, 0).unwrap(); assert_eq!(values.first().unwrap(), &i);
not really testing the percpu-ness of it
test/integration-test/src/tests/hash_map.rs line 225 at r8 (raw file):
for i in 0_u32..5 { let key = i.pow(2); assert!(matches!(hash_map.get(&key, 0), Err(MapError::KeyNotFound)));
assert!(matches!(...)) is an inferior version of assert_matches! because it doesn't show you the thing that didn't match.
test/integration-test/src/tests/hash_map.rs line 231 at r8 (raw file):
let key = i.pow(2); let values = hash_map.get(&key, 0).unwrap(); assert_eq!(values.first().unwrap(), &i);
not really testing the percpu-ness of it
ebpf/aya-ebpf/src/maps/hash_map.rs line 110 at r7 (raw file):
); hash_map!( "docs/lru_hash_map.md",
I guess these can be shared with the legacy maps, but the example is probably going overboard -- that md will not be used beyond right here, so you could just inline it.
…map` * The methods of all structs are almost identical, use a macro to reduce code repetition. * Use the third person in all docstrings.. * Make use of `#[doc]` and split out the most repetetive chunks into separate files. * Make the `Safety` comment for `get*` operations more clear, provide context and inks.
Introduce BTF map definitions for hash maps, with custom structs indicating the metadata of the map, which end up in the `.maps` section.
cd76d8a to
92f914f
Compare
|
@vadorovsky, this pull request is now in conflict and requires a rebase. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduce BTF map definitions for hash maps, with custom structs indicating the metadata of the map, which end up in the
.mapssection.This change is