Describe the bug
Upgrading KubeBlocks from 1.0.3-beta.5 to 1.0.3-beta.15 triggers rolling recreation of existing database Pods, even though the database Cluster.spec remains unchanged and no restart operation is requested.
The behavior was observed across multiple clusters in the test environment, including components configured with podUpdatePolicy: PreferInPlace.
To Reproduce
- Deploy KubeBlocks
1.0.3-beta.5.
- Create database clusters and wait until their components are
Running.
- Upgrade KubeBlocks to
1.0.3-beta.15 without modifying the existing Cluster specifications or requesting a restart.
- Observe InstanceSet reconciliation and database Pod deletion/recreation after the new operator starts.
Expected behavior
An operator-only upgrade should preserve running database Pods when users have not requested workload changes or a restart. Changes to operator-injected Pod configuration should have a backward-compatible migration strategy that avoids an automatic database rollout.
Actual behavior
Existing database Pods are deleted and recreated during reconciliation after the operator upgrade. The observed Cluster generation remains unchanged during this rollout.
This introduces unexpected database restarts and potential availability impact during an operator upgrade.
Additional context
A suspected contributor is #10201, which changes the generated kbagent Pod configuration by:
- Adding the
kubeblocks-role-label DownwardAPI volume.
- Mounting it at
/etc/kubeblocks/pod-metadata.
- Updating role-probe configuration in
KB_AGENT_PROBE.
These changes affect Pod template fields that cannot be updated in place and can trigger recreation despite PreferInPlace.
The upgrade also changes KubeBlocks-managed tools image tags, but image changes alone should not be assumed to explain Pod recreation. A complete before/after InstanceSet template comparison is needed to isolate all contributing changes.
Affected upgrade path: 1.0.3-beta.5 → 1.0.3-beta.15.
Describe the bug
Upgrading KubeBlocks from
1.0.3-beta.5to1.0.3-beta.15triggers rolling recreation of existing database Pods, even though the databaseCluster.specremains unchanged and no restart operation is requested.The behavior was observed across multiple clusters in the test environment, including components configured with
podUpdatePolicy: PreferInPlace.To Reproduce
1.0.3-beta.5.Running.1.0.3-beta.15without modifying the existing Cluster specifications or requesting a restart.Expected behavior
An operator-only upgrade should preserve running database Pods when users have not requested workload changes or a restart. Changes to operator-injected Pod configuration should have a backward-compatible migration strategy that avoids an automatic database rollout.
Actual behavior
Existing database Pods are deleted and recreated during reconciliation after the operator upgrade. The observed Cluster generation remains unchanged during this rollout.
This introduces unexpected database restarts and potential availability impact during an operator upgrade.
Additional context
A suspected contributor is #10201, which changes the generated kbagent Pod configuration by:
kubeblocks-role-labelDownwardAPI volume./etc/kubeblocks/pod-metadata.KB_AGENT_PROBE.These changes affect Pod template fields that cannot be updated in place and can trigger recreation despite
PreferInPlace.The upgrade also changes KubeBlocks-managed tools image tags, but image changes alone should not be assumed to explain Pod recreation. A complete before/after InstanceSet template comparison is needed to isolate all contributing changes.
Affected upgrade path:
1.0.3-beta.5→1.0.3-beta.15.