Skip to content

Make persistent sessions co-exist with remote cache feature#30859

Merged
mhajas merged 2 commits into
keycloak:mainfrom
ahus1:is-30855-coexist-persistent-sessions-external-infinispan
Jul 9, 2024
Merged

Make persistent sessions co-exist with remote cache feature#30859
mhajas merged 2 commits into
keycloak:mainfrom
ahus1:is-30855-coexist-persistent-sessions-external-infinispan

Conversation

@ahus1

@ahus1 ahus1 commented Jun 27, 2024

Copy link
Copy Markdown
Member

Closes #30855

Results with 50 logins+logouts per second, single-site ROSA, single site Aurora -> almost no change in the response times

Single user (so all other data is in memory), 3 pods, 4 CPU requested, 5 min run, ~1,5 vCPU per pod, Aurora 16.1

./benchmark.sh eu-west-1 --scenario=keycloak.scenario.authentication.AuthorizationCode --server-url=${KEYCLOAK_URL} --realm-name=test-realm --ramp-up=20 --measurement=300 --clients-per-realm=1 --users-per-realm=1 --users-per-sec=50 --logout-percentage=100 --log-http-on-failure

Scenario: volatile / persistent / persistent nocache

CPU usage database: 13 / 33 / 36

Commit count: 38 / 250 / 360

image

image

image

Closes keycloak#30855

Signed-off-by: Alexander Schwartz <aschwart@redhat.com>
@ahus1 ahus1 force-pushed the is-30855-coexist-persistent-sessions-external-infinispan branch from ada874d to ecd4142 Compare July 2, 2024 10:51
@ahus1 ahus1 marked this pull request as ready for review July 2, 2024 12:47
@ahus1 ahus1 requested review from a team as code owners July 2, 2024 12:47
@ahus1

ahus1 commented Jul 2, 2024

Copy link
Copy Markdown
Member Author

@mhajas - this is now ready for review. The load tests I did show that the performance is both good for no-cache and also no-batch variant. No-batch is considered a feature flag we might use for experimenting in KCB, so there are no additional tests for it.

@ahus1 ahus1 requested a review from mhajas July 2, 2024 12:50

@mhajas mhajas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR @ahus1, I added a few comments. Otherwise it LGTM

Closes keycloak#30855

Signed-off-by: Alexander Schwartz <aschwart@redhat.com>

@mhajas mhajas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @ahus1!

@mhajas mhajas merged commit d70f780 into keycloak:main Jul 9, 2024
Ladas added a commit to Ladas/kagenti-demo-deployment that referenced this pull request Nov 11, 2025
Problem:
- Keycloak operator still adding --optimized to args despite unsupported.podTemplate override
- StatefulSet showed duplicate jgroups args and --optimized flag
- Operator merges args instead of replacing them

Root Cause:
- Keycloak operator's unsupported.podTemplate only merges args, doesn't replace
- Operator always adds --optimized when it detects "optimized build"
- Need to override the entire command to take full control

Solution:
- Added `command: [/opt/keycloak/bin/kc.sh]` to podTemplate
- This prevents operator from injecting its default command+args
- Args now fully controlled by our CR specification
- Removed duplicate jgroups arg

Expected Result:
- StatefulSet will have exactly: command=[kc.sh] args=[jgroups, start, --verbose]
- No --optimized flag
- Keycloak will start successfully on first boot

Ref: keycloak/keycloak#30859

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Ladas added a commit to Ladas/kagenti-demo-deployment that referenced this pull request Nov 11, 2025
Problem:
- Keycloak operator ALWAYS adds --optimized flag to container args
- The --optimized flag requires pre-built cache (kc.sh build)
- On fresh installation, Keycloak exits with error instead of building cache
- Operator's unsupported.podTemplate.containers.command/args overrides do NOT work
  * Tested: command override - operator still added --optimized
  * Tested: args override - operator merged (didn't replace) and still added --optimized
  * Root cause: Operator bug in v26.0.7

Solution - InitContainer Workaround:
- Added initContainer that runs `kc.sh build --features=spiffe:v1`
- InitContainer creates optimized cache in shared emptyDir volume
- Main Keycloak container mounts same volume at /opt/keycloak/providers
- When main container starts with --optimized, cache already exists
- No more --optimized flag errors!

Changes:
1. Added initContainer: keycloak-build
   - Runs kc.sh build with SPIFFE feature enabled
   - Shares keycloak-providers volume with main container
2. Removed command/args overrides (they don't work anyway)
3. Main container inherits built cache from initContainer

Result:
- InitContainer runs first, builds cache
- Main container starts with --optimized flag (added by operator)
- Keycloak starts successfully because cache exists

Trade-offs:
- Adds ~5-10s to pod startup time (initContainer build phase)
- But this is acceptable for reliable deployment
- Better than CrashLoopBackOff!

Ref: keycloak/keycloak#30859

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Ladas added a commit to Ladas/kagenti-demo-deployment that referenced this pull request Nov 11, 2025
Problem:
- Keycloak Operator ALWAYS adds --optimized flag when starting Keycloak
- --optimized requires pre-built cache (via 'kc.sh build')
- Build-time options (health-enabled, SPIFFE) require running build before start
- Without init container: Keycloak fails "don't use --optimized for first startup"
- WITH init container: Operator copies it to realm import Jobs, causing build errors

Root Cause:
- Keycloak Operator copies ENTIRE unsupported.podTemplate to ALL pods
- This includes StatefulSet pods AND realm import Job pods
- No way to selectively apply pod template only to StatefulSet

Solutions Attempted:
1. ❌ Init container workaround → Inherited by Jobs, causing build config errors
2. ❌ Pod-level annotations → Also inherited by Jobs
3. ❌ Namespace-level sidecar disable → Jobs still got sidecars (label priority)

Temporary Solution (this commit):
- DISABLE build-time options (health-enabled, SPIFFE features)
- Keycloak will start successfully with --optimized and no build required
- Realm import Jobs will also work (no init container to inherit)
- Platform will be functional for development/testing

Trade-offs:
- ❌ No health endpoint (minor - can use readiness probe)
- ❌ No SPIFFE/SPIRE workload identity (can add later)
- ✅ Keycloak StatefulSet starts successfully
- ✅ Realm import Jobs complete successfully
- ✅ Platform functional end-to-end

Future Enhancements (documented in keycloak-cr.yaml):
1. Build custom Keycloak image with features pre-built
2. Use MutatingWebhook to add sidecar.istio.io/inject: false to Job pods
3. Contribute patch to Keycloak Operator for pod template overrides

References:
- Keycloak Operator --optimized issue: keycloak/keycloak#30859
- Istio Jobs lifecycle issue: istio/istio#11659

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make persistent user sessions and external Infinispan co-exist

3 participants