Skip to content

Repair wp/6.9 branch Actions: Remove flaky test issue reporting job - #83286

Draft
desrosj wants to merge 1 commit into
fix-actions-6.9/pin-wp-version-performance-testsfrom
fix-actions-6.9/disable-flaky-test-issue-reporting
Draft

desrosj wants to merge 1 commit into
fix-actions-6.9/pin-wp-version-performance-testsfrom
fix-actions-6.9/disable-flaky-test-issue-reporting

Conversation

@desrosj

@desrosj desrosj commented Sep 21, 2026

Copy link
Copy Markdown
Member

Part of #82785.

What?

Removes the report-to-issues job from .github/workflows/end2end-test.yml, along with the two things that existed only to feed it: the has-flaky-test-report output on merge-artifacts, and the id: merge-flaky-tests-reports on the step that output read from.

Nothing else changes. The e2e jobs still archive their flaky test reports, and merge-artifacts still merges them into a downloadable flaky-tests-report artifact.

wp/6.9 is the last branch still carrying this job; every other maintained wp/X.Y branch has already had it removed.

Why?

The job checks out live trunk — not this branch — and then runs ./packages/report-flaky-tests from that checkout. It therefore always executes against whatever the action looks like on trunk today, rather than against a fixed, frozen target. That makes it a permanent source of drift on a frozen branch, and trunk has since moved twice:

  1. #81218 (2026-08-06) rewrote the action to report flaky tests as a pull request comment and stop creating GitHub issues, dropping the label input.
  2. #82249 (2026-09-01) unified the automation comments, dropping repo-token as well and adding a required output-path input.

The drift is already visible in this branch's own CI. The most recent run of this job on wp/6.9 (run 31178713984, 2026-08-07) logged:

##[warning]Unexpected input(s) 'label', valid inputs are ['repo-token', 'artifact-path']

Since then trunk's packages/report-flaky-tests/action.yml has narrowed further, to:

inputs:
    artifact-path:
    output-path:
        required: true

So all three inputs this job passes are now wrong: repo-token and label are both unexpected, and the required output-path is never supplied. Trunk's src/run.ts reads it with core.getInput( 'output-path', { required: true } ) after it has collected the flaky tests, and src/index.ts routes a thrown error through core.setFailed(). Because the job only runs at all when a flaky test report exists, the next flaky test on this branch takes that path and fails the job outright.

Even setting the hard failure aside, the job can no longer do the thing it is named for: trunk's action only renders markdown to a file now. It does not open [Type] Flaky Test issues, which is the behavior this branch's workflow was written to invoke.

Retrofitting trunk's current design was considered and rejected. It is not just a CI repair — it is a product-policy change (flaky tests become PR-comment-only and are never filed as issues) and it would pull a rewritten, type-stripped Node 24 action onto a frozen, security-maintenance-only branch. Removing the job drops only an automated notification; no test pass/fail signal depends on it, and the flaky test data is still published as an artifact.

packages/report-flaky-tests itself is left in place. After this change no workflow on this branch uses it, but removing the package is a separate, optional cleanup.

Use of AI Tools

This PR was created by Claude Code under my supervision. All code should be treated as AI-produced and not yet reviewed by a human until this PR is marked Ready for Review.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8

`report-to-issues` checks out live `trunk` rather than this branch before running
`./packages/report-flaky-tests`, so it always executes against whatever that action
looks like on trunk today instead of a fixed target. Trunk has since changed the
action's shape twice, and the job's inputs no longer match it: the most recent run
of this job on `wp/6.9` already logged `Unexpected input(s) 'label'`, and trunk's
`action.yml` now accepts only `artifact-path` and a required `output-path` that this
job never supplies. `repo-token` and `label` are both gone.

The rewrite behind those changes is also a product-policy change, not a CI fix:
flaky tests are now rendered into a pull request comment and are no longer filed as
GitHub issues at all. Porting that to a frozen, security-maintenance-only branch is
out of scope, so the job is removed instead. Doing so drops only an automated
notification; no test pass/fail signal depends on it, and the flaky test artifacts
are still collected and merged by the jobs above.

The `has-flaky-test-report` output on `merge-artifacts` and the step `id` feeding it
existed only to gate this job, so they are removed with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtSAAJkQ9P6vPtDumqStg8
@desrosj desrosj added the [Type] Build Tooling Issues or PRs related to build tooling label Sep 21, 2026
@coderabbitai

coderabbitai Bot commented Sep 21, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@desrosj
desrosj added this pull request to stack #82877 September 21, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Type] Build Tooling Issues or PRs related to build tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant