feat(ci): add Firefox/WebKit smoke coverage with named projects - #166
Merged
Conversation
- Add named chromium, firefox, webkit projects to playwright.config.js - Keep bypassCSP: true in chromium project only - Update test:vrt, test:a11y, test:api scripts to --project=chromium - Add test/vrt/browser-smoke.spec.js (non-screenshot smoke tests) - Update CI smoke step to run browser-smoke.spec.js under firefox/webkit - Remove continue-on-error from smoke step (fail-closed) - Remove generated report file from tracking
…x CI order - Rename all Chromium VRT snapshots to include -chromium (baseline migration) - Rewrite smoke spec: assert HTTP 200, fixture-specific content, viewport - Move Chromium install before test:api (cold runner fix) - Delete tracked report file (will be recreated by reporter) - Update test/vrt/README.md with full CI gate documentation
The smoke spec navigates directly via page.goto() and does not use vrt-helpers.loadFixture(). Remove the unused import to clean up.
- Move setViewportSize before page.goto in smoke spec (P2) - Make snapshot-update docs Chromium-only (P1: prevents Firefox/WebKit baseline generation)
Playwright was writing HTML reports to test/vrt/reports/html, but CI uploads only test/vrt/results/. Update the HTML reporter outputFolder to test/vrt/results/html so generated output lands inside the git- ignored results/ directory that CI uploads on failure. Update README test structure to reflect the unified results/ path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Firefox and WebKit smoke coverage to the CI quality gate for card #129, with named Playwright projects and fail-closed smoke testing.
Changes
playwright.config.js
chromium,firefox,webkitprojectsbypassCSP: trueonly in chromium project (Chromium-specific workaround)bypassCSPpackage.json
test:vrt,test:a11y,test:apiall pinned to--project=chromium.github/workflows/ci.yml
test:api(fixes cold runner: API test needs browser)browser-smoke.spec.jsunder--project=firefox --project=webkit --workers 1continue-on-error— smoke step is fail-closedtest/vrt/browser-smoke.spec.js (new)
test/vrt/README.md
--project=chromium(prevents Firefox/WebKit baseline generation)test/vrt/snapshots/
-chromiumsuffix (baseline migration for named projects)test/vrt/reports/html/index.html
Evidence
-chromiumsnapshot namingcontinue-on-erroron any gateStandards Compliance
test/vrt/reports/html--project=chromium --update-snapshots)