Skip to content

Allow UserStorageProvider implementations to opt into storing service account users - #49804

Open
jbutturini wants to merge 8 commits into
keycloak:mainfrom
jbutturini:service-account-user-storage
Open

Allow UserStorageProvider implementations to opt into storing service account users#49804
jbutturini wants to merge 8 commits into
keycloak:mainfrom
jbutturini:service-account-user-storage

Conversation

@jbutturini

Copy link
Copy Markdown

Adds an optional UserServiceAccountProvider capability interface so a UserStorageProvider can own service account users, instead of UserStorageManager always persisting them to local JPA storage. Opt-in and backward compatible: realms without a provider implementing the interface are unaffected, and existing local service accounts still resolve (local storage is checked first). Motivation and use cases are in the issue and discussion #49802.

Includes an integration test (ServiceAccountUserStorageTest) and a test provider implementing the interface.

Happy to adjust the SPI shape if a different pattern is preferred.


AI usage: the initial implementation was drafted with the assistance of an AI agent and reviewed/validated by the contributors before submission.

Closes #49803

@jbutturini
jbutturini requested a review from a team as a code owner June 8, 2026 21:46
@jbutturini

jbutturini commented Jun 10, 2026

Copy link
Copy Markdown
Author

@mposolda @pedroigor would appreciate a maintainer's eyes on this when you have the cycles to take a look. Heads up that #49839 (by @tanmayg1502) implements the same idea against the same issue (#49803). I'm the issue author and happy to converge, their version has a couple of nice ideas I'd gladly fold in (notably UserServiceAccountProvider extends UserRegistrationProvider, and external-first read semantics in getServiceAccount). Happy to go with whichever direction you prefer so we land one clean version. Do you have a preference on the contract and test layout between the two PRs?

@jbutturini

Copy link
Copy Markdown
Author

Friendly nudge on this, it's been a couple of weeks since it landed in the team/core-iam queue (thanks @stianst for routing it!). @mposolda @pedroigor @ahus1 the main thing blocking a merge is probably the overlap with #49839, so I want to make that decision as low-effort as possible for you. Both PRs implement the same UserServiceAccountProvider interface for #49803. I'm happy to do whichever is easiest to review:

  1. Converge into a single PR: I'll can fold in Allow external storage for service account users #49839's ideas and we close the other, or
  2. Whichever PR you'd rather take as the base, I'll adapt mine to match.

Just let me know the direction and I'll turn it around quickly!

Copilot AI balanced review requested due to automatic review settings June 28, 2026 21:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an opt-in SPI capability (UserServiceAccountProvider) allowing UserStorageProvider implementations to own service account users, instead of always persisting them to local JPA storage. It updates UserStorageManager routing accordingly while keeping backwards compatibility by resolving existing local service accounts first.

Changes:

  • Add UserServiceAccountProvider (server-spi-private) for service account creation/lookup delegation to external user storage.
  • Update UserStorageManager to route service-account user creation/lookup to enabled providers (and adjust federated-storage cleanup behavior).
  • Add an integration test plus a custom in-memory provider to validate external storage routing behavior.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/custom-providers/src/main/resources/META-INF/services/org.keycloak.storage.UserStorageProviderFactory Registers the new custom test provider factory.
tests/custom-providers/src/main/java/org/keycloak/tests/providers/federation/ServiceAccountUserStorageFactory.java Adds an in-memory test UserStorageProviderFactory backing the integration test.
tests/custom-providers/src/main/java/org/keycloak/tests/providers/federation/ServiceAccountUserStorage.java Implements a test UserStorageProvider that opts into owning service account users.
tests/base/src/test/java/org/keycloak/tests/federation/storage/ServiceAccountUserStorageTest.java Adds integration coverage for external service-account storage behavior.
server-spi-private/src/main/java/org/keycloak/storage/user/UserServiceAccountProvider.java Introduces the opt-in capability interface for service account ownership.
model/storage/src/main/java/org/keycloak/storage/adapter/AbstractUserAdapterFederatedStorage.java Updates Javadoc to allow overriding service account client-link persistence in adapters.
model/storage/src/main/java/org/keycloak/storage/adapter/AbstractUserAdapter.java Updates Javadoc to allow overriding service account client-link persistence in adapters.
model/storage-private/src/main/java/org/keycloak/storage/UserStorageManager.java Routes service-account add/lookup to providers and adjusts federated-storage cleanup conditions.

@sampromises

sampromises commented Jul 28, 2026

Copy link
Copy Markdown

Hey @mposolda @pedroigor @ahus1, reviving this on behalf of my tech lead -- could one of you please take a look at this?

We've been working this year to migrate our company's tech stack to Keycloak; and this PR remains a key part from launching Client support thru Keycloak in the next month.

@ahus1 ahus1 self-assigned this Aug 5, 2026
Copilot AI review requested due to automatic review settings August 5, 2026 05:10
jbutturini and others added 2 commits August 5, 2026 14:11
… account users

Service account users (service-account-<clientId>) are currently always
persisted in local JPA storage, even when a realm delegates all other user
management to an external UserStorageProvider. UserStorageManager hard-codes
the local-storage path for these accounts, which creates a split-brain for
deployments backed by an external user store.

This adds an optional UserServiceAccountProvider capability interface that a
UserStorageProvider may implement to handle service account creation and
lookup. UserStorageManager checks for a provider implementing the interface
before falling back to local storage. The change is purely additive and
opt-in: realms without a qualifying provider are unaffected, and existing
locally-stored service accounts remain resolvable since local storage is
still checked first.

Closes keycloak#49803

Co-authored-by: nlajeunesse <lajeunessenicholas@gmail.com>
Signed-off-by: Jonathan Butturini <jbutturini.developer@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Alexander Schwartz <alexander.schwartz@gmx.net>
@ahus1
ahus1 force-pushed the service-account-user-storage branch from dd94357 to 4b9457c Compare August 5, 2026 05:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Not ready to approve

The SPI omits mandatory removal and rename requirements, and external lookup lacks established timeout and failover handling.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (1)

model/storage-private/src/main/java/org/keycloak/storage/UserStorageManager.java:1071

  • This external lookup bypasses the timeout and graceful-degradation path used by other federated user lookups (tryResolveFederatedUser). A slow or StorageUnavailableException-throwing high-priority provider can therefore fail/block service-account resolution, including client-credentials grants, instead of continuing to the next provider. Use mapEnabledStorageProvidersWithTimeout and handle unavailability consistently.
        return getEnabledStorageProviders(client.getRealm(), UserServiceAccountProvider.class)
                .map(provider -> provider.getServiceAccount(client))
                .filter(Objects::nonNull)
                .findFirst()
  • Files reviewed: 8/8 changed files
  • Comments generated: 2
  • Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Copilot AI review requested due to automatic review settings August 5, 2026 05:16
Signed-off-by: Alexander Schwartz <alexander.schwartz@gmx.net>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The SPI omits mandatory removal and rename lifecycle requirements, and the reference provider fails client-ID updates.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (3)

server-spi-private/src/main/java/org/keycloak/storage/user/UserServiceAccountProvider.java:33

  • The lookup contract described here does not match the implementation: UserStorageManager.getServiceAccount checks local storage first and only then queries capability providers. Document creation and lookup ordering separately so providers do not assume they can override an existing local service account.
 * <p>All storage providers that implement this interface will be looped through in priority order.
 * If a method returns null, the next provider is tried. If no provider handles the request,
 * local storage is used as a fallback.

server-spi-private/src/main/java/org/keycloak/storage/user/UserServiceAccountProvider.java:37

  • UserRegistrationProvider is mandatory for an external model, not optional: UserStorageManager.removeUser resolves the model's provider as that capability and throws a ModelException when it is absent. Either make this interface include a removal contract or state the requirement as mandatory.
 * <p>Providers implementing this interface should also implement {@link UserRegistrationProvider}
 * so that service account users can be removed when the owning client is deleted or has service
 * accounts disabled.

server-spi-private/src/main/java/org/keycloak/storage/user/UserServiceAccountProvider.java:50

  • The returned model must also support durable username changes: ClientManager.clientIdChanged unconditionally calls setUsername on the service-account user. Without this contract, a provider can satisfy the documented methods but make ordinary client-ID updates fail.
     * <p>The returned {@link UserModel} must support {@link UserModel#setEnabled(boolean)} and
     * {@link UserModel#setServiceAccountClientLink(String)}, and must durably persist the client
     * link so that it survives across sessions and is returned by
     * {@link UserModel#getServiceAccountClientLink()} on subsequent lookups.
  • Files reviewed: 8/8 changed files
  • Comments generated: 1
  • Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Signed-off-by: Alexander Schwartz <alexander.schwartz@gmx.net>
Copilot AI review requested due to automatic review settings August 5, 2026 05:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The capability contract omits mandatory lifecycle capabilities, and the test provider fails when a client ID is changed.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (3)

server-spi-private/src/main/java/org/keycloak/storage/user/UserServiceAccountProvider.java:51

  • The returned model must also support setUsername: changing a client's ID unconditionally renames its service-account user in ClientManager.clientIdChanged. Without this contract, a conforming provider may fail ordinary client updates.
     * <p>The returned {@link UserModel} must support {@link UserModel#setEnabled(boolean)} and
     * {@link UserModel#setServiceAccountClientLink(String)}, and must durably persist the client
     * link so that it survives across sessions and is returned by
     * {@link UserModel#getServiceAccountClientLink()} on subsequent lookups.

tests/custom-providers/src/main/java/org/keycloak/tests/providers/federation/ServiceAccountUserStorage.java:62

  • Renaming any client with an externally stored service account currently fails because the client update path always calls setUsername. Update this test provider to move the map entry and keep the adapter's username current, then cover client-ID changes in the integration test.
            @Override
            public void setUsername(String username) {
                throw new UnsupportedOperationException();

server-spi-private/src/main/java/org/keycloak/storage/user/UserServiceAccountProvider.java:40

  • This capability currently permits providers that cannot complete the lifecycle it initiates: provider-qualified users are removed only through UserRegistrationProvider and resolved by ID only through UserLookupProvider, otherwise deletion throws and later lookups return null. Make these mandatory capabilities (or route both operations through this interface).
 * <p>Providers implementing this interface should also implement {@link UserRegistrationProvider}
 * so that service account users can be removed when the owning client is deleted or has service
 * accounts disabled.
 */
public interface UserServiceAccountProvider {
  • Files reviewed: 8/8 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Signed-off-by: Alexander Schwartz <alexander.schwartz@gmx.net>
Copilot AI review requested due to automatic review settings August 5, 2026 06:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

External identity stability and cache reload compatibility must be corrected, and federated cleanup needs effective coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (2)

server-spi-private/src/main/java/org/keycloak/storage/user/UserServiceAccountProvider.java:39

  • Implementing only this advertised capability is insufficient when the user cache is enabled. UserCacheSession.findServiceAccount caches the returned external ID and later reloads it through getUserById, while UserStorageManager.getUserById only consults UserLookupProvider; after cache invalidation/partial eviction, a provider implementing this interface alone can no longer resolve its service account. Either require UserLookupProvider here or adapt the service-account cache reload path to use this capability.
public interface UserServiceAccountProvider extends UserRegistrationProvider {

model/storage-private/src/main/java/org/keycloak/storage/UserStorageManager.java:523

  • The new external-service-account preRemove branch is not exercised by the added deletion test: that test only checks the provider's map, which is removed regardless of whether federated sidecar cleanup ran. Add a federated attribute/role/group mapping before deletion and assert its row is gone afterward, so reverting this condition would fail the test.
        if (getFederatedStorage() != null
                && (user.getServiceAccountClientLink() == null || !StorageId.isLocalStorage(user.getId()))) {
            getFederatedStorage().preRemove(realm, user);
  • Files reviewed: 8/8 changed files
  • Comments generated: 1
  • Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Copilot AI review requested due to automatic review settings August 5, 2026 06:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The capability is duplicated in the public SPI, and local-first backward compatibility lacks direct integration coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (1)

model/storage-private/src/main/java/org/keycloak/storage/UserStorageManager.java:1067

  • Add integration coverage for the promised local-first compatibility path: create a local service account, then enable the capability provider and verify lookup returns the original local user rather than the external provider. The current fallback test removes the provider before creation, so it would not catch a regression in this ordering.
        UserModel user = localStorage().getServiceAccount(client);
        if (user != null) return user;
        return getEnabledStorageProviders(client.getRealm(), UserServiceAccountProvider.class)
  • Files reviewed: 9/9 changed files
  • Comments generated: 1
  • Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

* <p>This capability extends {@link UserRegistrationProvider} because service account users must
* be removable when the owning client is deleted or has service accounts disabled.
*/
public interface UserServiceAccountProvider extends UserRegistrationProvider {
Copilot AI review requested due to automatic review settings August 5, 2026 06:51
@ahus1
ahus1 force-pushed the service-account-user-storage branch from 7e80ce2 to 92632e3 Compare August 5, 2026 06:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The duplicated public SPI, incomplete lookup contract, and unbounded external lookup must be addressed.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (3)

server-spi/src/main/java/org/keycloak/storage/user/UserServiceAccountProvider.java:39

  • The linked issue and both discussions explicitly keep this capability in server-spi-private so it is not a stable public SPI. Defining the same FQCN here also publishes it from keycloak-server-spi and leaves duplicate class definitions across the two artifacts; remove this public copy and retain only the private-SPI declaration.
public interface UserServiceAccountProvider extends UserRegistrationProvider {

model/storage-private/src/main/java/org/keycloak/storage/UserStorageManager.java:1071

  • This direct provider call bypasses the storage-provider timeout wrapper used by federated lookups (tryResolveFederatedUser, lines 1298–1309). A slow external store can therefore stall service-account lookup—and client-credentials requests—without the configured timeout handling; use mapEnabledStorageProvidersWithTimeout here.
        return getEnabledStorageProviders(client.getRealm(), UserServiceAccountProvider.class)
                .map(provider -> provider.getServiceAccount(client))
                .filter(Objects::nonNull)
                .findFirst()
                .orElse(null);

tests/base/src/test/java/org/keycloak/tests/admin/userstorage/ServiceAccountUserStorageTest.java:148

  • This exercises only the default transient client-credentials session, so it never verifies that the externally stored user can be reloaded by ID. Add a case enabling USE_REFRESH_TOKEN_FOR_CLIENT_CREDENTIALS_GRANT and refreshing the token; otherwise the persistent-session path required for external service accounts can regress unnoticed.
        // client_credentials grant works against the externally-stored service account.
        oauth.client(CLIENT_ID, CLIENT_SECRET);
        AccessTokenResponse response = oauth.doClientCredentialsGrantAccessTokenRequest();
  • Files reviewed: 8/8 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Copilot AI review requested due to automatic review settings August 5, 2026 06:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The capability is exposed from the wrong SPI module, and external lookup bypasses configured timeout and failover handling.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (1)

server-spi/src/main/java/org/keycloak/storage/user/UserServiceAccountProvider.java:39

  • The linked issue explicitly lists promotion to the public server-spi as a non-goal and says this experimental contract should live in server-spi-private. Defining it here exposes a public compatibility commitment before the contract is proven; move the interface to the private SPI module and update consumers accordingly.
public interface UserServiceAccountProvider extends UserRegistrationProvider {
  • Files reviewed: 8/8 changed files
  • Comments generated: 1
  • Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Comment on lines +1067 to +1070
return getEnabledStorageProviders(client.getRealm(), UserServiceAccountProvider.class)
.map(provider -> provider.getServiceAccount(client))
.filter(Objects::nonNull)
.findFirst()
ahus1 added 3 commits August 5, 2026 16:08
Signed-off-by: Alexander Schwartz <alexander.schwartz@gmx.net>
Signed-off-by: Alexander Schwartz <alexander.schwartz@gmx.net>
Signed-off-by: Alexander Schwartz <alexander.schwartz@gmx.net>
Copilot AI review requested due to automatic review settings August 5, 2026 07:09
@ahus1
ahus1 force-pushed the service-account-user-storage branch from 92632e3 to b3252bc Compare August 5, 2026 07:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The capability contract lacks required ID lookup support, and provider lookup bypasses established failure handling.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (2)

model/storage-private/src/main/java/org/keycloak/storage/UserStorageManager.java:1071

  • This new external lookup bypasses the storage-provider timeout and graceful-degradation path used by regular external user lookups (tryResolveFederatedUser at lines 1298-1310). If the highest-priority provider is unavailable, client-credentials requests fail instead of continuing to the next provider; use the timeout helper and handle StorageUnavailableException consistently.
        return getEnabledStorageProviders(client.getRealm(), UserServiceAccountProvider.class)
                .map(provider -> provider.getServiceAccount(client))
                .filter(Objects::nonNull)

server-spi-private/src/main/java/org/keycloak/storage/user/UserServiceAccountProvider.java:42

  • The capability does not require UserLookupProvider, although service-account IDs are subsequently resolved via getUserById (including cache delegate reloads in UserAdapter#getUserModel). A provider implementing only this declared contract can create and return an account initially, but later updates or ID-based resolution fail; make lookup part of the capability contract, as the test provider already assumes.
public interface UserServiceAccountProvider extends UserRegistrationProvider {
  • Files reviewed: 8/8 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow service account users to be stored in external user storage providers

6 participants