Repository navigation
Missing labels datas on some metrics since k8s v1.37.0 #3142
Description
Hi,
Since I moved to k8s 1.37 (as of writing, k8s 1.37.1, cri-o 1.37.2) some labels are missing, e.g. in container_network_receive_bytes_total :
container_network_receive_bytes_total{container="",id="/",image="",interface="cilium_host",name="",namespace="",pod=""} 2440 1791376932773
container_network_receive_bytes_total{container="",id="/",image="",interface="cilium_net",name="",namespace="",pod=""} 95060 1791376932773
container_network_receive_bytes_total{container="",id="/",image="",interface="eno1",name="",namespace="",pod=""} 4.6865022608e+10 1791376932773
container_network_receive_bytes_total{container="",id="/",image="",interface="lxc0ffbf568a9a9",name="",namespace="",pod=""} 8.871278e+06 1791376932773
container_network_receive_bytes_total{container="",id="/",image="",interface="lxc1cb10c48f4f6",name="",namespace="",pod=""} 2.985128e+06 1791376932773
container_network_receive_bytes_total{container="",id="/",image="",interface="lxc2f265225e358",name="",namespace="",pod=""} 5.05903415e+08 1791376932773
container_network_receive_bytes_total{container="",id="/",image="",interface="lxc3fbd406ec764",name="",namespace="",pod=""} 2686 1791376932773
container_network_receive_bytes_total{container="",id="/",image="",interface="lxc5bda1155d4f8",name="",namespace="",pod=""} 20477 1791376932773
container_network_receive_bytes_total{container="",id="/",image="",interface="lxc63bdb4cd127e",name="",namespace="",pod=""} 8.3999685e+07 1791376932773
I did try to set in kubelet and cri-o the cgroup driver to cgroupfs, and labels came back, but with some side effects and I revert the change, this cgroup driver is not recommended on systemd systems
For the moment, I only see that on network datas
Activity
kubernetes-prow commentedon Oct 9, 2026
This issue is currently awaiting triage.
If kube-state-metrics contributors determine this is a relevant issue, they will accept it by applying the
triage/acceptedlabel and provide further guidance.The
triage/acceptedlabel can be added by org members by writing/triage acceptedin a comment.Details
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
Nemric commentedon Oct 12, 2026
Well, it seems like kube v1.37 won't expose pods and containers metrics to cadvisor anymore
Then, you can enable the
PodAndContainerStatsFromCRIfeature gate in kubelet configIf you're using Cri-O, you can add
to your crio.conf file