Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for an undesired partition migration with stale leader epoch #4680

Merged
merged 4 commits into from
Apr 18, 2024

Conversation

emasab
Copy link
Contributor

@emasab emasab commented Apr 8, 2024

A partition migration could happen,
using stale metadata, when the partition
was undergoing a validation and
being retried because of an error.
Solved by doing a partition migration
only with a non-stale leader epoch.
Happening since 2.1.0

@emasab emasab force-pushed the dev_fix_undesired_partition_migration branch from f13b6d2 to 2779b09 Compare April 8, 2024 19:09
@emasab emasab force-pushed the dev_fix_metadata_fast_refresh_loop branch from ced675f to 74b82fc Compare April 11, 2024 09:18
@emasab emasab force-pushed the dev_fix_undesired_partition_migration branch from 2779b09 to 4b005f0 Compare April 11, 2024 09:19
@emasab emasab force-pushed the dev_fix_metadata_fast_refresh_loop branch from 74b82fc to 05c2830 Compare April 11, 2024 17:17
@emasab emasab force-pushed the dev_fix_undesired_partition_migration branch from 4b005f0 to b9e2442 Compare April 11, 2024 17:17
@emasab emasab force-pushed the dev_fix_metadata_fast_refresh_loop branch from 05c2830 to 8090e71 Compare April 14, 2024 11:03
@emasab emasab force-pushed the dev_fix_undesired_partition_migration branch from b9e2442 to fd0238d Compare April 14, 2024 11:04
@emasab emasab force-pushed the dev_fix_metadata_fast_refresh_loop branch from 8090e71 to bdccc8c Compare April 15, 2024 08:06
@emasab emasab force-pushed the dev_fix_undesired_partition_migration branch from fd0238d to 3c7d054 Compare April 15, 2024 08:07
@emasab emasab force-pushed the dev_fix_metadata_fast_refresh_loop branch from bdccc8c to f0d63ab Compare April 15, 2024 09:32
@emasab emasab force-pushed the dev_fix_undesired_partition_migration branch from 3c7d054 to f3972d1 Compare April 15, 2024 09:41
@emasab emasab force-pushed the dev_fix_metadata_fast_refresh_loop branch from f0d63ab to f9b1fa7 Compare April 15, 2024 09:44
@emasab emasab force-pushed the dev_fix_undesired_partition_migration branch from f3972d1 to cd88a71 Compare April 15, 2024 09:45
@emasab emasab force-pushed the dev_fix_metadata_fast_refresh_loop branch from f9b1fa7 to 7bfe82b Compare April 15, 2024 10:10
@emasab emasab force-pushed the dev_fix_undesired_partition_migration branch from cd88a71 to addd3d7 Compare April 15, 2024 10:11
@emasab emasab force-pushed the dev_fix_metadata_fast_refresh_loop branch from 7bfe82b to e3bc31e Compare April 15, 2024 14:51
@emasab emasab force-pushed the dev_fix_undesired_partition_migration branch from addd3d7 to 1411693 Compare April 15, 2024 14:51
@emasab emasab force-pushed the dev_fix_metadata_fast_refresh_loop branch from e3bc31e to 44dc011 Compare April 15, 2024 15:26
@emasab emasab force-pushed the dev_fix_undesired_partition_migration branch 2 times, most recently from b605960 to 7e06fa5 Compare April 15, 2024 17:24
Base automatically changed from dev_fix_metadata_fast_refresh_loop to dev_kip848 April 15, 2024 17:56
@emasab emasab force-pushed the dev_fix_undesired_partition_migration branch from 7e06fa5 to 92eb02b Compare April 15, 2024 18:02
with stale leader epoch

A partition migration could happen,
using stale metadata, when the partition
was undergoing a validation and
being retried because of an error.
Solved by doing a partition migration
only with a non-stale leader epoch.
Happening since 2.1.0
@emasab emasab force-pushed the dev_fix_undesired_partition_migration branch from 92eb02b to d56573d Compare April 17, 2024 17:49
Copy link
Contributor

@milindl milindl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for adding the test too

@emasab emasab merged commit 202bac9 into dev_kip848 Apr 18, 2024
2 checks passed
@emasab emasab deleted the dev_fix_undesired_partition_migration branch April 18, 2024 07:26
emasab added a commit that referenced this pull request Apr 18, 2024
A partition migration could happen,
using stale metadata, when the partition
was undergoing a validation and
being retried because of an error.
Solved by doing a partition migration
only with a non-stale leader epoch.
Happening since 2.1.0
anchitj pushed a commit that referenced this pull request Jun 10, 2024
A partition migration could happen,
using stale metadata, when the partition
was undergoing a validation and
being retried because of an error.
Solved by doing a partition migration
only with a non-stale leader epoch.
Happening since 2.1.0
kphelps added a commit to kphelps/librdkafka that referenced this pull request Oct 10, 2024
kphelps added a commit to DataDog/librdkafka that referenced this pull request Oct 10, 2024
This reverts commit 6584ed7.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants