Skip to content

osd: updated blocking pdbs if drained node is back #16506

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
travisn merged 1 commit into from
Sep 18, 2025
Merged

osd: updated blocking pdbs if drained node is back #16506

travisn merged 1 commit into from
Sep 18, 2025
+28 −2

Conversation

@sp98
Copy link
Contributor

@sp98 sp98 commented Sep 18, 2025 •
edited
Loading

If the drained node is added back and one or more others nodes are down, then update the blocking pdbs, that is, add blocking pdb to node that was just added back.

Issue resolved by this Pull Request:
Resolves #15840

Testing:

1. Current PDB state. 
 kr get pdb
NAME                MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
rook-ceph-mgr-pdb   N/A             1                 0                     71m
rook-ceph-mon-pdb   N/A             1                 1                     72m
rook-ceph-osd       N/A             1                 1                     2m20s

2. Drain `minikube` 
kubectl drain --ignore-daemonsets --force  --delete-emptydir-data -v=8 minikube

3. Current PDB state: 
❯ kr get pdb
NAME                              MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
rook-ceph-mgr-pdb                 N/A             1                 1                     74m
rook-ceph-mon-pdb                 N/A             1                 0                     75m
rook-ceph-osd-host-minikube-m02   N/A             0                 0                     6s
rook-ceph-osd-host-minikube-m03   N/A             0                 0                     6s

4. Stop minikube-03. 
 kr get nodes
NAME           STATUS                     ROLES           AGE     VERSION
minikube       Ready,SchedulingDisabled   control-plane   97m     v1.30.0
minikube-m02   Ready                      <none>          96m     v1.30.0
minikube-m03   NotReady                   <none>          7m59s   v1.30.0

5. Uncordon minikube
❯ kr get nodes
NAME           STATUS     ROLES           AGE     VERSION
minikube       Ready      control-plane   97m     v1.30.0
minikube-m02   Ready      <none>          96m     v1.30.0
minikube-m03   NotReady   <none>          8m48s   v1.30.0

6. Current PDB state 
❯ kr get pdb
NAME                              MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
rook-ceph-mgr-pdb                 N/A             1                 1                     78m
rook-ceph-mon-pdb                 N/A             1                 0                     79m
rook-ceph-osd-host-minikube       N/A             0                 0                     24s
rook-ceph-osd-host-minikube-m02   N/A             0                 0                     4m8s

7. Start minikube-03 
kr get nodes
NAME           STATUS   ROLES           AGE    VERSION
minikube       Ready    control-plane   101m   v1.30.0
minikube-m02   Ready    <none>          100m   v1.30.0
minikube-m03   Ready    <none>          17s    v1.30.0


8. Current PDB state:
❯ kr get pdb
NAME                MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
rook-ceph-mgr-pdb   N/A             1                 1                     81m
rook-ceph-mon-pdb   N/A             1                 1                     82m
rook-ceph-osd       N/A             1                 1                     7s

Checklist:

  • Commit Message Formatting: Commit titles and messages follow guidelines in the developer guide.
  • Reviewed the developer guide on Submitting a Pull Request
  • Pending release notes updated with breaking and/or notable changes for the next minor release.
  • Documentation has been updated, if necessary.
  • Unit tests have been added, if necessary.
  • Integration tests have been added, if necessary.

@sp98 sp98 force-pushed the osd-pdb-update-fd branch from 24753db to 9a3746f Compare September 18, 2025 07:33
@sp98
osd: updated blocking pdbs if drained node is back
f2bf0eb 
If the drained node is added back and one or more others nodes are down
update the blocking pdbs, that is, add blocking pdb to node that was
just added back.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
@sp98 sp98 force-pushed the osd-pdb-update-fd branch from 9a3746f to f2bf0eb Compare September 18, 2025 07:35
@sp98 sp98 requested a review from travisn September 18, 2025 14:23
travisn
travisn approved these changes Sep 18, 2025
View reviewed changes
Copy link
Member

@travisn travisn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@travisn travisn merged commit e6a8a0c into rook:master Sep 18, 2025
55 of 56 checks passed
@mergify mergify bot mentioned this pull request Sep 18, 2025
Merged
6 tasks
travisn added a commit that referenced this pull request Sep 18, 2025
@travisn
Merge pull request #16509 from rook/mergify/bp/release-1.18/pr-16506
ff780f6 
osd: updated blocking pdbs if drained node is back (backport #16506)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@travisn travisn travisn approved these changes

Assignees

No one assigned

Labels

backport-release-1.18

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Blocking PDBs won't be updated when failure domains have changed

2 participants

@sp98 @travisn