ci: retain test output and generate JUnit artifacts in Prow jobs by madhoshyagnik · Pull Request #10348 · cri-o/cri-o

madhoshyagnik · 2026-09-13T09:46:05Z

What type of PR is this?

/kind ci

What this PR does / why we need it:

In CRI-O's Prow jobs driven by playbooks in contrib/test/ci (ci-fedora-integration and ci-e2e), test output was discarded from live logs and JUnit XML reports were not generated or displayed in Prow:

In ci-fedora-integration, make localintegration 2>&1 > /tmp/artifacts/testout.txt redirected stdout to a file and only streamed stderr into the live log. Furthermore, Bats was not invoked with JUnit report flags, so Prow's Spyglass JUnit lens showed 0 test results.
In ci-e2e (and e2e-features.yml), kubetest was invoked with &> {{ artifacts }}/e2e.log, causing Ansible task failures to output "stdout": "" and hiding Ginkgo test progress and failure details from the build logs.

This PR addresses this by:

Updating test/test_runner.sh to generate JUnit reports (junit_parallel.xml and junit_serial.xml) into $REPORT_DIR or $ARTIFACT_DIR when set and supported by Bats (--report-formatter junit -o <dir>), while running both parallel and serial suites and preserving failure exit codes.
Adding ARTIFACT_DIR: "{{ artifacts }}" to int_test_env in contrib/test/ci/vars.yml so integration tests write JUnit XML files into the artifact collection path.
Updating contrib/test/ci/integration.yml to pipe make localintegration output to tee {{ artifacts }}/testout.txt with set -o pipefail.
Updating contrib/test/ci/e2e.yml and contrib/test/ci/e2e-features.yml to stream kubetest output via tee {{ artifacts }}/e2e.log with set -o pipefail and executable: /bin/bash.
Adding junit*.xml to .gitignore.

Which issue(s) this PR fixes:

Fixes #10190

Special notes for your reviewer:

None

Does this PR introduce a user-facing change?

None

Summary by CodeRabbit

Tests
- Improved end-to-end and integration test output by displaying results live while also saving complete logs as artifacts.
- Test failures are now reported reliably after all test suites finish, including when tests run in parallel.
- Added support for generating JUnit reports in the configured artifact directory when supported.
Chores
- Expanded ignored test-report patterns to prevent generated JUnit files from being included in version control.
- Standardized artifact directory configuration across integration and end-to-end test runs.

Currently, CRI-O's Prow integration and e2e jobs run via Ansible playbooks in contrib/test/ci. In ci-fedora-integration, stdout was redirected to a file while only stderr was kept in the live log, and Bats did not produce JUnit XML reports. In ci-e2e, kubetest stdout and stderr were redirected to e2e.log with '&>', resulting in empty output in Ansible task logs upon failure. This change: 1. Updates test/test_runner.sh to generate JUnit reports when REPORT_DIR or ARTIFACT_DIR is set, writing junit_parallel.xml and junit_serial.xml while preserving both parallel and serial exit codes. 2. Adds ARTIFACT_DIR to int_test_env in contrib/test/ci/vars.yml so Bats writes JUnit XML reports to the artifacts directory. 3. Updates contrib/test/ci/integration.yml to stream make localintegration output with tee and set -o pipefail. 4. Updates contrib/test/ci/e2e.yml and e2e-features.yml to stream kubetest output via tee with set -o pipefail and bash executable. 5. Adds junit*.xml to .gitignore. Signed-off-by: Madhosh Yagnik <madhosh1yagnik@gmail.com>

openshift-ci · 2026-09-13T09:46:12Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: madhoshyagnik
Once this PR has been reviewed and has the lgtm label, please assign giuseppe for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci · 2026-09-13T09:46:19Z

Hi @madhoshyagnik. Thanks for your PR.

I'm waiting for a cri-o member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

coderabbitai · 2026-09-13T09:46:25Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 5fc3dd6f-00a3-4e5b-a043-7a4d1ca15b77

📥 Commits

Reviewing files that changed from the base of the PR and between 0393a49 and 565914a.

📒 Files selected for processing (6)

.gitignore
contrib/test/ci/e2e-features.yml
contrib/test/ci/e2e.yml
contrib/test/ci/integration.yml
contrib/test/ci/vars.yml
test/test_runner.sh

Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.

📝 Walkthrough

Walkthrough

CI test commands now stream output to artifact logs and preserve pipeline failures. The test runner can emit separate parallel and serial JUnit reports, runs both suites after a failure, and returns the captured failure status.

Changes

CI artifact preservation

Layer / File(s)	Summary
Stream CI test output `contrib/test/ci/e2e-features.yml`, `contrib/test/ci/e2e.yml`, `contrib/test/ci/integration.yml`, `contrib/test/ci/vars.yml`	E2E and integration commands use Bash `pipefail` and `tee`. E2E tasks write output to the log while displaying it. Integration tests use the configured artifact path and define `ARTIFACT_DIR`.
Generate JUnit reports and complete both test runs `test/test_runner.sh`, `.gitignore`	The test runner writes separate parallel and serial JUnit reports when Bats supports the formatter. It runs both suites after a failure and returns the captured status. Git ignores the generated JUnit files.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 56591

The CI changes preserve live output, failure statuses, and JUnit artifacts for the targeted jobs, so the change is mergeable.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely summarizes the primary changes: retaining test output and generating JUnit artifacts in Prow jobs.
Linked Issues check	✅ Passed	The changes satisfy the coding objectives in `#10190`. `contrib/test/ci/integration.yml` uses `set -o pipefail` and `tee` to write integration output to `{{ artifacts }}/testout.txt` while keeping it in…
Out of Scope Changes check	✅ Passed	The changed files support the objectives in `#10190`. The artifact environment setting and `.gitignore` rule support report creation and artifact handling. The playbook and test-runner changes implement…
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

madhoshyagnik requested a review from mrunalp as a code owner September 13, 2026 09:46

openshift-ci Bot added release-note-none Denotes a PR that doesn't merit a release note. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. kind/ci Categorizes issue or PR as related to CI labels Sep 13, 2026

openshift-ci Bot requested review from QiWang19 and littlejawa September 13, 2026 09:46

openshift-ci Bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Sep 13, 2026

Navigation Menu

Repository navigation

ci: retain test output and generate JUnit artifacts in Prow jobs - #10348

ci: retain test output and generate JUnit artifacts in Prow jobs#10348
madhoshyagnik wants to merge 1 commit into
cri-o:mainfrom
madhoshyagnik:fix-prow-ci-test-artifacts

madhoshyagnik commented Sep 13, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

openshift-ci Bot commented Sep 13, 2026

Uh oh!

openshift-ci Bot commented Sep 13, 2026

Uh oh!

coderabbitai Bot commented Sep 13, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

madhoshyagnik commented Sep 13, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Summary by CodeRabbit

Uh oh!

openshift-ci Bot commented Sep 13, 2026

Uh oh!

openshift-ci Bot commented Sep 13, 2026

Uh oh!

coderabbitai Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

madhoshyagnik commented Sep 13, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Sep 13, 2026 •

edited

Loading