try and make tests more stable#50995
Conversation
There was a problem hiding this comment.
Pull request overview
Improves Admin UI Playwright test isolation, selectors, readiness checks, environment configuration, and feature handling.
Changes:
- Isolates group and SAML test data.
- Hardens table, alert, switch, and drag-and-drop interactions.
- Adds configurable server credentials and OID4VCI gating.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
test/utils/table.ts |
Hardens table selectors and extraction. |
test/utils/masthead.ts |
Extends alert readiness checks. |
test/utils/form.ts |
Adds switch interaction fallbacks. |
test/utils/constants.ts |
Adds environment-based server credentials. |
test/utils/AdminClient.ts |
Applies shared configuration and realm-scoped cleanup. |
test/groups/list.spec.ts |
Isolates group tests by realm and UUID. |
test/clients/saml.ts |
Handles SAML post-binding interaction failures. |
test/clients/saml.spec.ts |
Uses uniquely targeted SAML clients. |
test/client-scope/oid4vci-client-scope.spec.ts |
Adds deterministic readiness and feature gating. |
test/autentication/flows.spec.ts |
Adds drag-and-drop fallback strategies. |
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (4)
js/apps/admin-ui/test/utils/table.ts:112
locator.count()does not auto-wait, unlike the previousclick(). If the toolbar is still rendering, every count can be zero and this helper immediately throws; wait for an actionable candidate (for example with an auto-waiting combined locator) before deciding to use the overflow fallback.
const exactButton = toolbar.getByRole("button", {
name: itemName,
exact: true,
});
if ((await exactButton.count()) > 0) {
js/apps/admin-ui/test/utils/constants.ts:6
- The environment override is not consistently applied:
test/identity-providers/main.ts:4-8,99-100still hard-codeshttp://localhost:8080. WithKEYCLOAK_SERVER_URLset, login/admin calls target one server while those provider flows target another, so all server URL consumers need to use this constant.
export const SERVER_URL = normalizeServerUrl(
process.env.KEYCLOAK_SERVER_URL ?? DEFAULT_SERVER_URL,
);
js/apps/admin-ui/test/clients/saml.ts:54
- Catching every error also hides missing controls, timeouts, and browser/page failures; the test can then save an unchanged form and pass on the success alert. Only skip the interaction when the control is explicitly disabled/read-only and let unexpected failures propagate.
try {
await switchOff(page, "#attributes\\.saml🍺force🍺post🍺binding");
} catch {
// Some generated SAML clients expose this setting as read-only.
}
js/apps/admin-ui/test/autentication/flows.spec.ts:207
- The test now becomes skipped whenever all drag strategies fail, including when the product's reorder behavior has regressed. Keep the retrying helper, but fail when it returns
falseand retain the success-notification assertion so this behavior remains covered.
test.skip(
!moved,
"Drag-and-drop reorder could not be triggered deterministically in this runtime.",
);
closes keycloak#50992 resolves #34 closes keycloak#50995 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
* State leakage/coupling: reduced in groups/SAML flows by scoped test data and improved targeting. * Selector brittleness: reduced via table + alert helper hardening. * Timing races: reduced via deterministic readiness assertions and switch handling. * Environment assumptions: addressed via env-configurable server/auth and OID4VCI feature gating. fixes: keycloak#50994 Signed-off-by: Erik Jan de Wit <erikjan.dewit@gmail.com>
Signed-off-by: Erik Jan de Wit <erikjan.dewit@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
js/apps/admin-ui/test/clients/assign-oid4vci-client-scope.ts:10
- This OID4VCI path is still unconditional, so the assignment test fails while looking for this option whenever the server feature is disabled. Apply the same
OID4VC_VCIgate added tooid4vci-client-scope.spec.tsbefore selecting the protocol so unsupported environments skip consistently.
await selectItem(page, "#kc-protocol", "OpenID for Verifiable Credentials");
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (2)
js/apps/admin-ui/test/utils/table.ts:71
- The final fallback leaves the table and can click an unrelated global link (for example, a matching sidebar item) while rows are still loading, because the preceding
count()calls do not wait. Keep this fallback row-scoped and wait for the row instead.
const exactLink = page.getByRole("link", { name: itemName, exact: true });
if ((await exactLink.count()) > 0) {
await exactLink.first().click();
return;
}
await page.getByRole("link", { name: itemName }).first().click();
js/apps/admin-ui/test/utils/form.ts:129
check()/uncheck()can throw after dispatching the change when the switch rerenders. Unconditionally clicking in the catch path can then toggle an already-correct switch back, so re-read its state before applying the fallback.
await clickSwitch(switchElement);
| for (const action of directAction) { | ||
| if ((await action.count()) > 0) { | ||
| await action.first().click(); | ||
| return; | ||
| } | ||
| } |
| await page.getByTestId("name").fill(clientScopeName); | ||
| await clickSaveButton(page); | ||
| await expect(page.getByText("Client scope created")).toBeVisible(); | ||
| await createOid4vciClientScope(page, clientScopeName); |
| export const SERVER_URL = normalizeServerUrl( | ||
| process.env.KEYCLOAK_SERVER_URL ?? DEFAULT_SERVER_URL, | ||
| ); |
data and improved targeting.
switch handling.
and OID4VCI feature gating.
fixes: #50994
Signed-off-by: Erik Jan de Wit erikjan.dewit@gmail.com