create helm chart for keycloak-operator - #42079
Conversation
There was a problem hiding this comment.
Thank you for the contribution.
I'm adding hold label for now as we need to discuss internally first if a Helm Chart for the Operator is something that we'd like to include in the project.
Just a few points if we decided to bring this further.
This PR is missing a linked issue as per our contributing guidelines.Never mind, I missed it in the desc.- We'd need to avoid having to maintain the Helm Chart resources manually. It'd be good to reuse as much existing/generated resources (Roles, Deployment, Service, Service Account, etc.) as possible to have single source of truth.
- We'd also need some tests for this, similarly to our OLM tests.
- Docs update would be also needed.
- As a follow-up, we'd need to update the release process to include the Helm Chart in https://github.com/keycloak/keycloak-k8s-resources.
Hey, Thanks for the review, of course i will wait until the decision is made, regarding your points
Do you mean you wish to reuse the k8s manifests from the repo?
Of course i agree, i offered the grafana/prometheus repositories as inspiration, i am not very knowledgeable about GitHub actions, so im open to any ideas from the community.
I agree, if a decision to proceed with this PR will be reached i will also write docs about Again Thanks for the review! |
|
@vmuzikar |
nissessenap
left a comment
There was a problem hiding this comment.
First of all, thanks for your work on this @KyriosGN0 .
A few minor comments about the helm chart.
Around release management of helm charts you can take a look here: https://github.com/grafana/grafana-operator/blob/56279f56f8b0a6796b9383cb193748b427eb78c2/.github/workflows/release.yaml#L15-L51
In the grafana-operator we are supporting both OCI and index.yaml, but initially we only supported OCI. My suggested would be to only support OCI packaging for the helm chart.
Here you can also see how we read data from existing yaml files and put into our helm chart. This way we automatically update the CRDs and roles.
https://github.com/grafana/grafana-operator/blob/56279f56f8b0a6796b9383cb193748b427eb78c2/Makefile#L65-L82
Sadly, I see the keycloak operator is built on Java so we can't just copy similar logic since it's not built on kubebuilder.
I got too much todo at work right now to commit any GHA stuff.
|
Please see #37636 (comment) |
Removed the hold label. The review of the community survey showed strong interest in helm charts - but mostly for Keycloak itself. We see an intermediate value in offering a helm chart for the operator, but would like to ensure that the implementation requires minimal syncing when changes are made to the operator - #37636 (comment) For example #46980 was refined to directly use our generated yaml files rather than having modified copies just for kustomize overlays.
What we're looking for is reuse, rather than mostly copies of the artifacts - that will be difficult to maintain. You don't have to use the keycloak-k8s-resources repo if the helm chart generation logic lives in the keycloak repo. You would only use the keycloak-k8s-resources if this ends up being some kind of external effort. |
|
@KyriosGN0 After some internal discussion, we'd like to go forward with the Helm Chart for the Operator. However, we need to take a different approach than this PR currently is – as already outlined by @shawkins in the comment above. We'd suggest to evaluate the possibility for Helm Chart generation via JOSDK: https://javaoperatorsdk.io/docs/documentation/operations/helm-chart/ |
Yes but we'll need to find the correct distribution channel for the Helm Chart, and update the release process for it. But that could be a follow-up. |
|
hey @vmuzikar @shawkins i found quarkus-helm plugin which integrated nicely with the operator sdk
Right now Chart.yaml's version defaults to ${project.version} (same as the operator) and there's no appVersion. That works fine for "one chart release per operator release" — which is the simplest model and matches what most operator charts do.
This PR generates a chart and a .tar.gz in CI, but doesn't push them anywhere. The end-user helm install snippet in docs/guides/operator/installation.adoc currently references a TBD , I saw some issues around using OCI registry in qaurkus-helm, so a regular http repo can also work |
d072e54 to
39b89db
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (4)
docs/guides/operator/installation.adoc:299
- This command contradicts the instruction to keep CRDs: deleting the cluster-wide kustomization deletes its four CRDs and can garbage-collect all corresponding custom resources. The path also differs from the valid
cluster-widekustomization used at line 236. Delete only the operator workload/RBAC manifest, not the kustomization that includes CRDs.
kubectl delete -k 'github.com/keycloak/keycloak-k8s-resources/kubernetes/cluster-wide?ref={version}'
docs/guides/operator/installation.adoc:310
helm.sh/resource-policy: keepcontrols deletion; it does not add Helm's release ownership metadata. Moreover, the release was already installed in the previous step, so it cannot be “re-installed” under the same name. Document an ownership-aware upgrade (for example,helm upgrade --take-ownershipon a supported Helm version) or keep these CRDs managed out-of-band.
After the chart is installed and the operator is healthy, you can leave CRD management as-is (out-of-band) or transfer ownership to the chart on the next upgrade by setting the `helm.sh/resource-policy: keep` annotation on the existing CRDs and re-installing with `crds.enabled=true`.
docs/guides/operator/installation.adoc:289
- This lookup is also unversioned, so readers of an older installation guide would see values for the latest chart instead of the release documented on the page. Pin the OCI chart to
{version}.
Run `helm show values oci://quay.io/keycloak/keycloak-operator-helm` for the full list.
docs/guides/operator/installation.adoc:307
- The migration install is unversioned as well, so following an archived guide can migrate directly to a newer operator release. Pin this command to the documented release.
helm install keycloak-operator oci://quay.io/keycloak/keycloak-operator-helm \
--namespace keycloak --set crds.enabled=false
| helm install keycloak-operator oci://quay.io/keycloak/keycloak-operator-helm \ | ||
| --namespace keycloak --create-namespace |
Signed-off-by: AvivGuiser <avivguiser@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (5)
docs/guides/operator/installation.adoc:301
- The migration leaves the namespaced
Role/RoleBindingand the controller-generated ClusterRoles/ClusterRoleBindings in place. Those resources are also part of the chart, sohelm installwill stop with an invalid ownership error when it encounters them. Delete all operator RBAC resources by their establishedapp.kubernetes.io/namelabel while preserving the CRDs.
kubectl delete deployment,service,serviceaccount keycloak-operator -n keycloak
kubectl delete clusterrole keycloak-operator-clusterrole
kubectl delete clusterrolebinding keycloak-operator-clusterrole-binding
docs/guides/operator/installation.adoc:312
helm.sh/resource-policy: keepcontrols deletion; it does not add Helm's ownership metadata. Also, the release already exists at this point, so re-installing the same name fails. Document an actual adoption command such ashelm upgrade ... --take-ownership(with its Helm version requirement), or the equivalent ownership annotations and label.
After the chart is installed and the operator is healthy, you can leave CRD management as-is (out-of-band) or transfer ownership to the chart on the next upgrade by setting the `helm.sh/resource-policy: keep` annotation on the existing CRDs and re-installing with `crds.enabled=true`.
.github/workflows/operator-ci.yml:172
- With
pipefail, these checks can fail with status 141 even when the resource exists:grep -qexits on its first match whileechois still writing the large CRD-filled render and receives SIGPIPE. Feed the captured output directly togrepso valid renders do not fail CI.
echo "$out" | grep -qE '^kind: Deployment$'
# Cluster-wide install binds the controllers via ClusterRoleBinding.
echo "$out" | grep -qE '^kind: ClusterRoleBinding$'
.github/workflows/operator-ci.yml:187
- Under
pipefail, an unexpectedly rendered CRD can makeechoreceive SIGPIPE aftergrep -qfinds the first match, causing the pipeline to evaluate false and bypass this failure branch. Use a here-string so this negative test reliably detects CRDs.
if echo "$out" | grep -qE '^kind: "?CustomResourceDefinition"?$'; then
echo "expected no CRDs with crds.enabled=false"; exit 1
fi
operator/README.md:40
- This says only CRDs are post-processed, but the script also rewrites every ServiceAccount subject namespace in generated RoleBindings and ClusterRoleBindings. Mention that transformation so maintainers know the generated chart is not otherwise purely declarative.
Helm chart configuration lives in `operator/src/main/resources/application.properties` under the `quarkus.helm.*` keys. CRDs are post-processed by `operator/scripts/post-process-helm-chart.sh` (wrapping them in a `{{ if .Values.crds.enabled }}` guard) — the rest is purely declarative.
Unreported flaky test detectedIf the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR. org.keycloak.testsuite.forms.MultipleTabsLoginTest#multipleTabsParallelLoginTestWithAuthSessionExpiredAndRegisterClickKeycloak CI - Forms IT (firefox) |
|
@KyriosGN0 thank you for the additional updates. Let me look things over again as I think this is close to being committed. We'll need some kind of release note, but those can be done as a follow-up after the release process has been updated. |
|
@KyriosGN0 I think everything looks good - but wanted to check will even oci installs see the target/helm/kubernetes/keycloak-operator/templates/NOTES.txt? They seem much focused on the quarkus development scenario. |
|
Hey @shawkins from the docs i think that OCI install will see the NOTES.txt |
shawkins
left a comment
There was a problem hiding this comment.
Hey @shawkins from the docs i think that OCI install will see the NOTES.txt
Ok, we'll capture a follow-up for improving that as well.
Thank you @KyriosGN0, approving this PR as it should be good for initial experimental state.
@vmuzikar @michalvavrik please given it a look.
| /^kind:/{k=$2} /^ name:/{n=$2} /^ namespace:/{ns=$2} | ||
| /^---/{if(k&&n)print k"/"ns"/"n; k=n=ns=""} | ||
| END{if(k&&n)print k"/"ns"/"n}' | sort | uniq -d) | ||
| if [ -n "$dupes" ]; then echo "duplicate resources rendered:"; echo "$dupes"; exit 1; fi |
There was a problem hiding this comment.
quarkiverse/quarkus-operator-sdk#1393 is already released https://github.com/quarkiverse/quarkus-operator-sdk/releases/tag/7.8.0, can you drop it then?
|
@michalvavrik i planned to do it once this is merged, do you want it done as part of this PR ? |
I don't mind either way. This PR LGTM, whatever works for others. I don't see a reason not to do it now, but you may easily know something I don't. It is maintainers call to merge this. |
|
Hey @vmuzikar, could you please weigh in on #42079 (comment) ? |
I am at least twice as heavy as @vmuzikar , but alright 🤷♂️ |
We're not using QOSDK 7.8.0 just yet, we need to wait for another Quarkus Platform release when it adopts it. |
vmuzikar
left a comment
There was a problem hiding this comment.
Thank you! Changes look good but we're missing release notes.
|
@shawkins Do we have an issue already for updating the release process? |
I think that is fine to do in a follow-up once the release process changes are hashed out.
Created: keycloak-rel/keycloak-rel#196 We'll also have a follow-up on the NOTES.txt |
vmuzikar
left a comment
There was a problem hiding this comment.
Ok, let's get this merged already.
This PR creates a Helm Chart that deploys the Keycloak operator and allows to also install the CRD via a flag
part of #37636
Im not that knowledgeable about GitHub Actions, but i think that doing something similar to the prometheus-community helm chart
or the grafana helm chart is a good starting point, for release i haven't found the github action for this this repo, but maybe we can do the same mechanism and update the CRD in this folder?