Skip to content

Use cleanup methods provided by framework - #53876

Open
ahus1 wants to merge 6 commits into
keycloak:mainfrom
ahus1:is-53874-cleanup-tests
Open

ahus1 wants to merge 6 commits into
keycloak:mainfrom
ahus1:is-53874-cleanup-tests

Conversation

@ahus1

@ahus1 ahus1 commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Closes #53874

Decisions and assumptions:

  • ManagedRealm.updateClientWithCleanup now restores added client attributes with an empty value, as a client update only merges attributes. Without this, attributes such as client.session.max.lifespan stayed on the shared test-app client and broke later tests.
  • The user notBefore set by user.admin().logout() can't be reset through a user update, so RefreshTokenTest marks the user as dirty to have it re-created. The test sets notBefore to the real time before applying a positive offset, so the leftover value would be harmless to other tests. The dirty marker is for hygiene only.
  • Tests that need the time offset in the Infinispan caches use @InjectTimeOffSet(enableForCaches = true) instead of calling InfinispanTimeUtil manually. The offset reset after each test also disables it again.
  • Resetting the offset with enableForCaches = true no longer fails if the cache time service was never enabled. The time offset endpoint now uses InfinispanTimeUtil.disableTestingTimeServiceIfEnabled, as set(0) on a fresh state previously threw an IllegalStateException on the server. The strict disableTestingTimeService stays for the model tests, where an unbalanced enable/disable should still fail.
  • Cleanups still run in registration order, which only matters if a test updates the same resource twice. A change to reverse order would affect all cleanups and is tracked in Test framework runs cleanups in registration order instead of reverse order #54941.
  • Mid-test timeOffSet.set(0) calls remain, as they are part of the test logic.

Closes keycloak#53874

Signed-off-by: Alexander Schwartz <alexander.schwartz@ibm.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two conversions defer restoration past points where the tests require the original configuration.

2 open findings
What changed in this PR

Replaces manual test cleanup with framework-managed lifecycle cleanup.

Changes:

  • Converts realm, client, user, and identity-provider rollback logic.
  • Removes redundant time-offset resets.
  • Adds builder methods and cleanup guidance.
File Description
OrganizationAuthenticationTest.java Removes manual time reset.
OID4VCRefreshCredentialTest.java Removes redundant offset cleanup.
RefreshTokenTimeoutsTest.java Migrates configuration rollback.
RefreshTokenTest.java Migrates realm, client, and user cleanup.
OfflineTokenSessionManagementTest.java Migrates session configuration cleanup.
OfflineTokenRefreshTest.java Migrates offline-token cleanup.
OfflineTokenBasicFlowTest.java Migrates client and signature cleanup.
AbstractJWTAuthorizationGrantTest.java Migrates consent configuration cleanup.
UserEmailTest.java Removes manual time reset.
DeclarativeUserTest.java Migrates internationalization cleanup.
ConcurrentLoginTest.java Migrates password-policy cleanup.
ResourcesRestServiceTest.java Migrates UMA configuration cleanup.
LinkedAccountsRestServiceTest.java Uses managed identity-provider updates.
WRITING_TESTS.md Documents cleanup APIs and time-offset lifecycle.
BEST_PRACTICES.md Adds cleanup best practices.
RealmBuilder.java Adds UMA configuration support.
ClientBuilder.java Adds client notBefore support.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Signed-off-by: Alexander Schwartz <alexander.schwartz@ibm.com>
@keycloak-github-bot

Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.adapter.servlet.SAMLClockSkewAdapterTest#testTokenTimeIsValid

Keycloak CI - Adapter IT Strict Cookies

org.jboss.arquillian.graphene.enricher.exception.PageObjectInitializationException: 
Can not instantiate Page Object 
class org.keycloak.testsuite.adapter.page.SalesPostClockSkewServlet
 declared in: 
org.keycloak.testsuite.adapter.servlet.SAMLClockSkewAdapterTest
...

Report flaky test

@keycloak-github-bot keycloak-github-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

ahus1

This comment was marked as outdated.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The new documentation lists nonexistent client profile and policy helper signatures.

0 open findings

2 resolved since last review
Previously missed (1)

In code that hasn't changed since last review

Low severity Document correct ManagedRealm update method signatures

test-framework/​docs/​WRITING_TESTS.md:244

These signatures do not exist: ManagedRealm.updateClientProfile and updateClientPolicy each accept a list of representations, not (name, update). Document the callable signatures so readers can use these helpers without compilation errors.

🧠 Review effort: Balanced

Signed-off-by: Alexander Schwartz <alexander.schwartz@ibm.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The conversions preserve test behavior while consistently delegating rollback to the framework lifecycle.

0 open findings

🧠 Review effort: Balanced

Signed-off-by: Alexander Schwartz <alexander.schwartz@ibm.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues require further changes.

No open issues 🎉

Review effort: Balanced

@ahus1
ahus1 marked this pull request as ready for review November 9, 2026 18:18
@ahus1
ahus1 requested a review from a team as a code owner November 9, 2026 18:18
@ahus1
ahus1 enabled auto-merge (squash) November 9, 2026 18:18

@lhanusov lhanusov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ahus1 thank you for the update, please take a look on my comment, besides that it looks good to me 👍

P.S.: please check all InifinispanTimeUtil additions in this PR, I may missed some

events.clear();
timeOffSet.set(0);
}
realm.cleanup().add(r -> runOnServer.run(InfinispanTimeUtil.disableTestingTimeService()));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lhanusov, good catch. Switched OfflineTokenRefreshTest and OfflineTokenSessionManagementTest to @InjectTimeOffSet(enableForCaches = true) and removed all manual InfinispanTimeUtil enable/disable calls. I checked the PR diff for further additions, and there are none. I also added a short note to BEST_PRACTICES.md.


RealmResource realmResource = adminClient.realm("test");
runOnServer.run(InfinispanTimeUtil.enableTestingTimeService());
realm.cleanup().add(r -> runOnServer.run(InfinispanTimeUtil.disableTestingTimeService()));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also here

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved in ad2524f


RealmResource realmResource = adminClient.realm("test");
runOnServer.run(InfinispanTimeUtil.enableTestingTimeService());
realm.cleanup().add(r -> runOnServer.run(InfinispanTimeUtil.disableTestingTimeService()));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also here

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved in ad2524f

final int IDLE_LIFESPAN = 900;

runOnServer.run(InfinispanTimeUtil.enableTestingTimeService());
realm.cleanup().add(r -> runOnServer.run(InfinispanTimeUtil.disableTestingTimeService()));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved in ad2524f

Relates to keycloak#53874

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Alexander Schwartz <alexander.schwartz@ibm.com>
Copilot AI balanced review requested due to automatic review settings November 17, 2026 10:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two tests reset the cache time service before initialization, triggering server-side IllegalStateException errors that break class setup and prevent refresh assertions from running.

2 open issues

Review effort: Balanced

Relates to keycloak#53874

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Alexander Schwartz <alexander.schwartz@ibm.com>
Copilot AI balanced review requested due to automatic review settings November 17, 2026 10:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the migration to framework-managed test cleanup.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

@ahus1
ahus1 requested a review from lhanusov November 17, 2026 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace manual try/finally realm config rollback with updateWithCleanup in tests

3 participants