test: real integration + E2E (Playwright) verification tiers - #1
Merged
Conversation
Adds the verification substrate the auto-fix loop will gate on: tests that exercise the genuine pipeline against a real database and a real browser, not in-memory fakes. A fix isn't "done" until it's green here, end to end. - Ravelin.IntegrationTests: WebApplicationFactory<Program> over the real middleware pipeline + a Testcontainers SQL Server (Respawn resets state between tests). Covers the dotnet ingestion endpoint end to end: valid key persists findings, bad/no key -> 401, malformed payload -> 400 (never an unhandled 500). - Ravelin.E2ETests: launches the real published app as its own Kestrel process against a Testcontainers SQL Server and drives it with headless Chromium (Playwright). Landing-page WASM boot + login -> dashboard. - Program.cs: expose `public partial class Program` for the factory. - CI: new integration-tests and e2e-tests jobs in security.yml (Docker is available on GitHub-hosted runners). Also lands the dogfood SCA adapter the integration tests exercise: DotnetListAdapter + /api/ingest/dotnet route + unit tests + the security.yml self-scan push step.
- build-test ran `dotnet test Ravelin.slnx`, which swept in the E2E project (no browsers/published app there) and failed it. Narrow that step to the unit test project; integration + E2E keep their own dedicated jobs. - E2E launched the published app without a working directory, so the content root resolved to the test cwd and /_framework/* (the WASM bundle) 404'd — the runtime never booted and no page rendered. Pin WorkingDirectory + ASPNETCORE_CONTENTROOT to the app folder. - Add browser diagnostics (failed responses, console errors, HTML snippet) so an E2E failure explains itself instead of surfacing as a bare timeout.
This was referenced Jun 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The auto-fix loop we're building (capture → Linear → agent) only ships a fix when tests prove it works end to end. That guarantee is worthless unless the test tiers are real. Today the repo has 64 pure-domain unit tests and zero integration/E2E — the test project couldn't even see the host. This PR builds the missing substrate.
Two tiers, both hitting the genuine pipeline — no mocks of the system under test, so a hollow test can't pass:
Integration tier —
tests/Ravelin.IntegrationTestsWebApplicationFactory<Program>runs the real middleware pipeline (API-key auth → rate limit → endpoint → adapter →ScanReconciler→ EF Core) against a real SQL Server via Testcontainers; Respawn resets domain data between tests.E2E tier —
tests/Ravelin.E2ETests/dashboard(real/api/auth/login→ JWT → localStorage → authenticated API call).Plumbing
Program.cs:public partial class Programso the factory can drive the host.security.yml: newintegration-testsande2e-testsjobs (Docker is available on GitHub-hosted runners; the E2E job publishes the host + installs Chromium).DotnetListAdapter+/api/ingest/dotnet+ unit tests + the self-scan push step) — these were already in the working tree and the branch needs them to compile.Verification
What to watch on the first run
WebApplicationFactory(integration) and as a published process (E2E) — depends on static web assets being present; this is the most likely first-run wrinkle.