What happened?

mutates the pod object in place (vs creating a copy first). That's illegal and can cause data races or potentially panics (the panic is unlikely here but I can't rule it out).

This line should be:

pod = updatedPod
...
kl.podManager.UpdatePod(pod)

This should be fixed because we might serialize the object in the middle of this from somewhere else and if the object changes midway the protobuf object we create may not be valid, leading to panics in kubelets with this gate turned on.

/sig node
/priority critical-urgent
/kind bug
/assign @vinaykul

What did you expect to happen?

No mutation of pod objects from within Sync*Pod methods (in fact no mutation of pod should occur in most loops).

How can we reproduce it (as minimally and precisely as possible)?

N/A

Anything else we need to know?

No response

Kubernetes version

$ kubectl version
# paste output here

Cloud provider

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here

Install tools

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)