Skip to content

cri: increase default 2s timeout for crictl commands#22214

Merged
medyagh merged 4 commits into
kubernetes:masterfrom
medyagh:crio_imagels
Dec 18, 2025
Merged

cri: increase default 2s timeout for crictl commands#22214
medyagh merged 4 commits into
kubernetes:masterfrom
medyagh:crio_imagels

Conversation

@medyagh

@medyagh medyagh commented Dec 18, 2025

Copy link
Copy Markdown
Member

attempt to fix this in KVM CRIO Linux Tests
where "image ls" hits the limit

W1217 22:03:37.118496   21489 cache_images.go:736] Failed to list images for profile functional-683881 crictl images: sudo crictl images --output json: Process exited with status 1
stdout:
stderr:
E1217 22:03:37.106555   10313 log.go:32] "ListImages with filter from image service failed" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" filter="image:{}"

#21754 (comment)

@k8s-ci-robot k8s-ci-robot requested review from nirs and prezha December 18, 2025 19:10
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Dec 18, 2025
@medyagh medyagh changed the title crictl: increase default 2s timeout for crictl commands containerd and crio: increase default 2s timeout for crictl commands Dec 18, 2025
@medyagh

medyagh commented Dec 18, 2025

Copy link
Copy Markdown
Member Author

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Dec 18, 2025
@afbjorklund

afbjorklund commented Dec 18, 2025

Copy link
Copy Markdown
Collaborator

You shouldn't have special code for docker, that seems like a legacy from the dockershim?

i.e. it should also call listCRIImages

Need to remove docker from the code...

@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Dec 18, 2025
@medyagh

medyagh commented Dec 18, 2025

Copy link
Copy Markdown
Member Author

You shouldn't have special code for docker, that seems like a legacy from the dockershim?

i.e. it should also call listCRIImages

Need to remove docker from the code...

thats a good idea, I can do that in a separate PR I think I wanna limit this PR to crictl only

@afbjorklund

afbjorklund commented Dec 18, 2025

Copy link
Copy Markdown
Collaborator

Was just reflecting on the "containerd and crio" subject, while the change is related to CRI (i.e. everything)

@medyagh medyagh changed the title containerd and crio: increase default 2s timeout for crictl commands cri: increase default 2s timeout for crictl commands Dec 18, 2025
@medyagh medyagh requested a review from Copilot December 18, 2025 19:31
@medyagh

medyagh commented Dec 18, 2025

Copy link
Copy Markdown
Member Author

Was just reflecting on the "containerd and crio" subject, while the change is related to CRI (i.e. everything)

thanks, changed the title, also created an issue for removing docker calls for CRI calls in the code base, will be tested in a sepaerate PR

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR increases the default timeout for crictl commands from 2 seconds to 10 seconds to fix timeout issues encountered in KVM CRIO Linux Tests, specifically where "image ls" commands were hitting the 2-second limit and failing with "context deadline exceeded" errors.

Key Changes:

  • Added a constant timeoutOverrideFlag = "--timeout=10s" to override crictl's default 2s timeout
  • Updated crictl commands in crictlList, getCRIInfo, and listCRIImages to use the new timeout flag
  • Updated test mocks and expectations to handle the new timeout parameter

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
pkg/minikube/cruntime/cri.go Added timeout constant and applied it to crictl ps, crictl info, and crictl images commands
pkg/minikube/cruntime/cruntime_test.go Updated fake crictl implementation to strip timeout flag before processing test commands
pkg/minikube/bootstrapper/bsutil/kubeadm_test.go Updated test expectations to include the timeout flag in the crictl info command
pkg/minikube/kubeconfig/testdata/kubeconfig/config1 Removed the preferences: {} line from test data (appears unrelated to the PR's purpose)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

current-context: minikube
kind: Config
preferences: {}
users: null

Copilot AI Dec 18, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change appears to be unrelated to the PR's purpose of increasing crictl command timeouts. The removal of the "preferences: {}" line from this kubeconfig test data should be in a separate PR or the PR description should explain why this change is necessary.

Suggested change
users: null
users: []

Copilot uses AI. Check for mistakes.
@minikube-pr-bot

Copy link
Copy Markdown

kvm2 driver with docker runtime

┌────────────────┬──────────┬────────────────────────┐
│    COMMAND     │ MINIKUBE │ MINIKUBE  ( PR 22214 ) │
├────────────────┼──────────┼────────────────────────┤
│ minikube start │ 41.5s    │ 40.1s                  │
│ enable ingress │ 15.8s    │ 16.0s                  │
└────────────────┴──────────┴────────────────────────┘
Details

Times for minikube start: 43.7s 40.9s 39.5s 42.2s 41.1s
Times for minikube (PR 22214) start: 43.3s 38.6s 39.5s 40.1s 39.0s

Times for minikube ingress: 15.7s 15.7s 16.2s 15.7s 15.7s
Times for minikube (PR 22214) ingress: 16.2s 16.2s 15.7s 15.7s 16.2s

docker driver with docker runtime

┌────────────────┬──────────┬────────────────────────┐
│    COMMAND     │ MINIKUBE │ MINIKUBE  ( PR 22214 ) │
├────────────────┼──────────┼────────────────────────┤
│ minikube start │ 22.8s    │ 24.0s                  │
│ enable ingress │ 10.6s    │ 10.6s                  │
└────────────────┴──────────┴────────────────────────┘
Details

Times for minikube (PR 22214) start: 20.5s 25.4s 24.6s 25.2s 24.2s
Times for minikube start: 25.5s 20.8s 25.2s 20.4s 21.9s

Times for minikube (PR 22214) ingress: 10.6s 10.6s 10.6s 10.6s 10.6s
Times for minikube ingress: 10.6s 10.6s 10.6s 10.6s 10.7s

docker driver with containerd runtime

┌────────────────┬──────────┬────────────────────────┐
│    COMMAND     │ MINIKUBE │ MINIKUBE  ( PR 22214 ) │
├────────────────┼──────────┼────────────────────────┤
│ minikube start │ 22.1s    │ 19.2s                  │
│ enable ingress │ 23.3s    │ 22.9s                  │
└────────────────┴──────────┴────────────────────────┘
Details

Times for minikube start: 21.2s 22.8s 22.3s 22.5s 21.6s
Times for minikube (PR 22214) start: 17.6s 20.5s 21.6s 17.8s 18.7s

Times for minikube ingress: 24.1s 23.2s 23.2s 23.1s 23.1s
Times for minikube (PR 22214) ingress: 23.1s 23.1s 23.1s 22.1s 23.1s

@k8s-ci-robot

Copy link
Copy Markdown
Contributor

@medyagh: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
integration-vfkit-docker-macos-arm64 eaf3d28 link false /test integration-vfkit-docker-macos-arm64
integration-docker-crio-linux-x86-64 eaf3d28 link true /test integration-docker-crio-linux-x86-64
integration-kvm-docker-linux-x86-64 eaf3d28 link true /test integration-kvm-docker-linux-x86-64
integration-kvm-crio-linux-x86-64 eaf3d28 link true /test integration-kvm-crio-linux-x86-64

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@medyagh

medyagh commented Dec 18, 2025

Copy link
Copy Markdown
Member Author

/lgtm

@k8s-ci-robot

Copy link
Copy Markdown
Contributor

@medyagh: you cannot LGTM your own PR.

Details

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@medyagh medyagh merged commit 5ba5796 into kubernetes:master Dec 18, 2025
38 of 58 checks passed
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: medyagh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@minikube-pr-bot

Copy link
Copy Markdown

Here are the number of top 10 failed tests in each environments with lowest flake rate.

Environment Test Name Flake Rate
KVM_Linux_containerd (1 failed) TestFunctionalNewestKubernetes/Versionv1.35.0-rc.1/serial/ComponentHealth(gopogh) Unknown
KVM_Linux_crio (10 failed) TestFunctionalNewestKubernetes/Versionv1.35.0-rc.1/parallel/DashboardCmd(gopogh) Unknown
KVM_Linux_crio (10 failed) TestFunctional/parallel/DashboardCmd(gopogh) 0.00% (chart)
KVM_Linux_crio (10 failed) TestFunctional/parallel/ServiceCmdConnect(gopogh) 0.00% (chart)
KVM_Linux_crio (10 failed) TestFunctional/parallel/ServiceCmd/DeployApp(gopogh) 0.00% (chart)
KVM_Linux_crio (10 failed) TestFunctional/parallel/ServiceCmd/HTTPS(gopogh) 0.00% (chart)
KVM_Linux_crio (10 failed) TestFunctional/parallel/ServiceCmd/Format(gopogh) 0.00% (chart)
KVM_Linux_crio (10 failed) TestFunctional/parallel/ServiceCmd/URL(gopogh) 0.00% (chart)
Docker_Linux_containerd (1 failed) TestAddons/parallel/Registry(gopogh) 4.00% (chart)
KVM_Linux (9 failed) TestISOImage/PersistentMounts//data(gopogh) Unknown
KVM_Linux (9 failed) TestISOImage/PersistentMounts//var/lib/docker(gopogh) Unknown
KVM_Linux (9 failed) TestISOImage/PersistentMounts//var/lib/cni(gopogh) Unknown
KVM_Linux (9 failed) TestISOImage/PersistentMounts//var/lib/kubelet(gopogh) Unknown
KVM_Linux (9 failed) TestISOImage/PersistentMounts//var/lib/minikube(gopogh) Unknown
KVM_Linux (9 failed) TestISOImage/PersistentMounts//var/lib/toolbox(gopogh) Unknown
KVM_Linux (9 failed) TestISOImage/PersistentMounts//var/lib/boot2docker(gopogh) Unknown
KVM_Linux (9 failed) TestISOImage/VersionJSON(gopogh) Unknown
KVM_Linux (9 failed) TestISOImage/eBPFSupport(gopogh) Unknown

Besides the following environments also have failed tests:

  • Docker_Linux_containerd_arm64: 34 failed (gopogh)

  • Docker_Linux_crio: 26 failed (gopogh)

  • Docker_Linux_crio_arm64: 56 failed (gopogh)

  • Docker_Windows: 34 failed (gopogh)

To see the flake rates of all tests by environment, click here.

star3am pushed a commit to star3am/minikube that referenced this pull request Jan 2, 2026
* fix: add 10s timeout to crictl images command to prevent hangs

* feat: Introduce a 10s timeout override flag and apply it to crictl ps, info, and images commands.

* feat: add timeout flag to `crictl info` commands and remove empty kubeconfig preferences.

* fix: properly handle `crictl --timeout` flag and remove empty preferences from test kubeconfig
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants