Repository navigation
Preserve workflow definitions in realm export and import - #53486
goutamadwant wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Large accepted schedule intervals can overflow the delay calculation, and the new authorization boundary lacks targeted coverage.
Review effort: Balanced
Findings: 1
Open (2)
What changed in this PR
Adds provider-aware realm export/import for workflow definitions while preserving security, compatibility, and scheduling behavior.
Changes:
- Adds dedicated workflow serialization and legacy component compatibility.
- Masks administrative-export secrets and enforces access restrictions.
- Defers timer registration until transaction commit and adds integration coverage.
| File | Description |
|---|---|
tests/base/src/test/java/org/keycloak/tests/workflow/WorkflowExportImportTest.java |
Tests workflow export, import, validation, and scheduling. |
tests/base/src/test/java/org/keycloak/tests/workflow/WorkflowExportImportDisabledTest.java |
Tests behavior with workflows disabled. |
services/src/main/java/org/keycloak/services/resources/admin/RealmAdminResource.java |
Restricts workflows in administrative exports. |
server-spi-private/src/test/java/org/keycloak/models/utils/StripSecretsUtilsTest.java |
Tests workflow-step secret masking. |
server-spi-private/src/main/java/org/keycloak/models/utils/StripSecretsUtils.java |
Masks declared workflow-step secrets. |
server-spi-private/src/main/java/org/keycloak/exportimport/ExportOptions.java |
Adds workflow inclusion configuration. |
model/storage-private/src/main/java/org/keycloak/storage/datastore/DefaultExportImportManager.java |
Routes workflow imports through their provider. |
model/storage-private/src/main/java/org/keycloak/exportimport/util/WorkflowExportImportUtils.java |
Converts dedicated and legacy workflow definitions. |
model/storage-private/src/main/java/org/keycloak/exportimport/util/ExportUtils.java |
Exports workflows separately from components. |
model/jpa/src/main/java/org/keycloak/models/workflow/DefaultWorkflowProvider.java |
Defers new timer registration until commit. |
docs/documentation/server_admin/topics/workflows/managing-workflows.adoc |
Documents export/import behavior and limitations. |
core/src/main/java/org/keycloak/representations/idm/RealmRepresentation.java |
Adds the workflows realm field. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| if (seconds <= 0 || seconds > Integer.MAX_VALUE) { | ||
| throw new ModelValidationException("Workflow schedule interval must be between 1 and " + Integer.MAX_VALUE + " seconds"); |
| // this means that if clients is true but groups/roles is false the service account is exported without roles | ||
| // the other option is just include service accounts if clientsExported && groupsAndRolesExported | ||
| ExportOptions options = new ExportOptions(false, clientsExported, groupsAndRolesExported, clientsExported, true); | ||
| options.setWorkflowsIncluded(auth.isRealmAdmin()); |
505e539 to
b6fdd42
Compare
|
Both comments are addressed in b6fdd42. Scheduling delays now use long arithmetic across local and cluster paths, and the new permission test covers manage-realm export without workflow definitions. Please take another look. Thanks! |
sguilhen
left a comment
There was a problem hiding this comment.
Overall: the direction is right, and the diagnosis is correct: DefaultWorkflowProviderFactory.getConfigProperties() returns List.of(), so StripSecretsUtils.stripComponentConfigMap treats every workflow config key as undeclared and deletes it. Routing workflows through WorkflowProvider instead of the generic component importer is the correct fix, and it matches how we keep other internal component types out of the component contract.
I verified the security boundary and found no new exposure:
options.setWorkflowsIncluded(auth.isRealmAdmin())mirrorsWorkflowsResource's constructor-levelauth.requireRealmAdmin(), so amanage-realmonly admin gains nothing new.- Both
WorkflowProviderFactoryandWorkflowStepProviderFactorydefaultisInternal()totrue, andComponentResourcerejects internal components on list, get, create, update and delete, so there is no alternative read path through the component API. - Workflows can still only be created through full realm import, which requires
create-realmor master admin.PartialImportManagernever touches components, so amanage-realmadmin cannot smuggle one in. - The new masking replaces the value list rather than mutating it, so the realm cache is not poisoned. Good that there is a test asserting exactly that.
- The new guards in
importComponentsare a genuine hardening win: a crafted export can no longer plant orphan step components.
Main things I would like addressed before merge are the legacy-import hard failure (comment on WorkflowExportImportUtils.fromComponent), and a rebase, since WorkflowValidator.validateSchedule landed in main on 12 Oct and makes part of this diff dead code. The rest are smaller.
Two process notes: the new files are missing the Apache license header (though neighbouring workflow files lack it too, so flagging rather than insisting), and this changes the export format in a user-visible way, so it needs an entry in docs/documentation/upgrading rather than only the server-admin topic.
| } | ||
|
|
||
| private static WorkflowRepresentation fromComponent(ComponentExportRepresentation component) { | ||
| if (!"default".equals(component.getProviderId()) || component.getConfig() == null || component.getConfig().isEmpty()) { |
There was a problem hiding this comment.
Must address. Existing realm exports stop importing entirely here.
An empty config is exactly what an affected version's admin partial export produces; the reproducer in #50972 shows "config": {} verbatim. Before this change those files imported fine: importComponentModel found the factory, the default validateConfiguration is a no-op, and an inert component was created. Now the whole realm import aborts, and the message does not tell the user which entry to remove.
The users most likely to be holding such a file are precisely the ones this PR is for, since that is the broken artifact they already have on disk.
Could we log a warning and skip the entry instead? If failing loudly is the deliberate choice, the message should at least name the component id and the remediation, something like: Workflow component <id> has no configuration; it was produced by an export affected by #50972 and cannot be restored. Remove it from the "components" section before importing.
| if (workflows.isEmpty()) { | ||
| return; | ||
| } | ||
| if (!Profile.isFeatureEnabled(Profile.Feature.WORKFLOWS)) { |
There was a problem hiding this comment.
Minor, mostly a docs point. Worth confirming the intent: with the feature disabled, ExportUtils deliberately keeps the legacy components in an offline export, but they can never be fed back in, because this throws.
To be clear, this is not a regression. WorkflowProviderFactory.isSupported unregisters the factory when the feature is off, so ComponentUtil.getComponentFactory already failed with No such provider 'default'. Your error is strictly better.
But the doc presents the retained components as a backup, so I think it needs to say the feature must be re-enabled before the export can be restored. See my comment on managing-workflows.adoc.
| return; | ||
| } | ||
| long seconds = DurationConverter.parseDuration(scheduled).toSeconds(); | ||
| if (seconds <= 0 || seconds > Integer.MAX_VALUE) { |
There was a problem hiding this comment.
Needs a rebase. This is now dead code. WorkflowValidator.validateSchedule landed in main in 74d9c74 (12 Oct, nine days after this commit) and already rejects both < 1s and > Integer.MAX_VALUE, and toModel calls validateWorkflow before addWorkflow, so control never reaches this branch.
It also throws a hard-coded English string, whereas the rest of this class uses message bundle keys (workflowCannotUpdateSupports, workflowStepsIncompatible, and so on) resolved via ErrorResponse.resolveMessage. The equivalent key, workflowScheduleAfterInvalid, already exists in messages_en.properties.
Please drop the check on rebase. The int to long change in computeInitialDelay should stay; that is a real overflow fix, since lastRunSecs + intervalSecs could wrap negative and schedule the task immediately.
testUnsupportedScheduleDoesNotPersist will still pass after removal, it will just be exercising the validator.
| }); | ||
| } | ||
|
|
||
| private void scheduleWorkflow(Workflow workflow) { |
There was a problem hiding this comment.
scheduleCreatedWorkflow is a near copy of this method with the deferral added, which leaves the fix applied to the create path only.
updateWorkflow's scheduled-steps branch still calls cancelScheduledWorkflow then scheduleWorkflow eagerly, so a rolled-back update leaves the timer either cancelled or rescheduled for configuration that was never committed. That is the same class of bug the PR is fixing for create.
Could this be one method with the enlistAfterCompletion deferral applied in all cases? That removes the duplication and covers updateWorkflow and rescheduleWorkflow too. notifyScheduleChange already defers correctly, so the cluster path has always been fine and the two would then be consistent.
|
|
||
| // components | ||
| MultivaluedHashMap<String, ComponentExportRepresentation> components = exportComponents(realm, realm.getId()); | ||
| if (Profile.isFeatureEnabled(Feature.WORKFLOWS) || options.isPartial() || !options.isWorkflowsIncluded()) { |
There was a problem hiding this comment.
Readability. This condition took me a few passes. It reduces to "keep the legacy components only on an offline export when the feature is off", and !options.isWorkflowsIncluded() is subsumed by options.isPartial() for every production caller, since RealmAdminResource is the only place that sets it and it always passes partial=true.
Could we invert it into something self-describing? For example:
// Workflows are exported through their provider; the stored components are only kept as an
// opaque backup in offline exports taken while the feature is disabled.
boolean keepLegacyComponents = !Profile.isFeatureEnabled(Feature.WORKFLOWS)
&& !options.isPartial() && options.isWorkflowsIncluded();
if (!keepLegacyComponents) {
components.remove(WorkflowProvider.class.getName());
}| } | ||
|
|
||
| importOrganizations(rep, newRealm); | ||
| WorkflowExportImportUtils.importWorkflows(session, newRealm, rep); |
There was a problem hiding this comment.
Completeness. This covers full realm import. Two sibling paths silently drop workflows instead:
POST /admin/realms/{realm}/partialImport, sincePartialImportManageronly registers clients, roles, IdPs, IdP mappers, groups and users. The doc calls this unsupported, which is fine, but silently discarding sits oddly next to the explicit "cannot silently discard" stance taken inimportWorkflows.PUT /admin/realms/{realm}, which goes throughRepresentationToModel.updateRealmand is not mentioned in the doc at all.
Security-wise both are fine, and in fact desirable, since it keeps workflow creation behind create-realm. I am only asking that we either reject explicitly or document the second one too.
| Exported definitions include workflow settings and step configuration, but not in-progress workflow executions. | ||
| Step identifiers are regenerated on import. Workflows are restored after the other realm resources, so restoring users | ||
| does not activate the imported workflows. The workflows feature must be enabled to import workflow definitions. | ||
| When the feature is disabled, command-line exports preserve stored workflow definitions in the legacy component format; |
There was a problem hiding this comment.
Suggest making the restore path explicit here, since the current wording reads as though the retained components are a usable backup:
When the feature is disabled, command-line exports preserve stored workflow definitions in the legacy component format; administrative exports omit them. Such an export can only be imported after the workflows feature is enabled again.
| Only realm administrators with the appropriate permissions can manage workflows as they are considered sensitive operations. | ||
| For more details, see <<_understanding_different_types_realm_admins_,Understanding different types of Realm Admins>>. | ||
|
|
||
| == Exporting and importing workflow definitions |
There was a problem hiding this comment.
This section is good, but I think the change also needs an entry in docs/documentation/upgrading, because it is visible to anyone with existing tooling:
- realm exports gain a top-level
workflowsarray; org.keycloak.models.workflow.WorkflowProviderno longer appears undercomponents;- realm JSON that previously imported can now be rejected (see my comment on
WorkflowExportImportUtils.fromComponent).
| .forEach((providerType, componentList)-> componentList | ||
| .forEach(component -> stripComponentExport(session, providerType, component, fnGetConfigProperties)))); | ||
|
|
||
| Optional.ofNullable(rep.getWorkflows()).ifPresent(workflows -> workflows.forEach(workflow -> |
There was a problem hiding this comment.
Nice that this also covers the realm-create admin event, since AdminEventBuilder.representation routes through stripSecrets.
One gap for a follow-up rather than this PR: WorkflowsResource.create and the update endpoint pass a bare WorkflowRepresentation to adminEvent.representation(...), and REPRESENTATION_FORMATTER has no entry for that type, so a declared step secret is masked in a realm export but stored in the clear in the workflow-create admin event. Pre-existing, but it is the same masking goal.
| // this means that if clients is true but groups/roles is false the service account is exported without roles | ||
| // the other option is just include service accounts if clientsExported && groupsAndRolesExported | ||
| ExportOptions options = new ExportOptions(false, clientsExported, groupsAndRolesExported, clientsExported, true); | ||
| options.setWorkflowsIncluded(auth.isRealmAdmin()); |
There was a problem hiding this comment.
Confirming this is the right predicate, since it is the security-relevant line in the PR: WorkflowsResource's constructor calls auth.requireRealmAdmin(), so gating on auth.isRealmAdmin() keeps the export exactly in step with the API that owns the data. testManageRealmExportOmitsWorkflows covers the boundary. No change requested.
sguilhen
left a comment
There was a problem hiding this comment.
Switching this to "request changes" to flag the blocking item explicitly. Full detail is in my previous review; nothing new here.
The one change I consider necessary before merge is the hard failure in WorkflowExportImportUtils.fromComponent on legacy workflow components with an empty config. That is the exact artifact an admin partial export produced on affected versions, so realm JSON that imports today would stop importing, and the error does not tell the user what to remove. Warning and skipping, or at minimum an actionable message naming the component id, would resolve it.
A rebase is also needed, since WorkflowValidator.validateSchedule landed in main in 74d9c74 and makes the schedule range check in scheduleCreatedWorkflow unreachable.
Everything else in that review is a suggestion rather than a blocker, and the security boundary checks out as posted.
|
Thanks, the follow up changes are prepared locally. Empty legacy workflow entries are skipped with a warning, and timer registration and cancellation happen after commit across creation, updates and rescheduling. I also removed validation already covered by WorkflowValidator, retained the long delay arithmetic, omitted empty workflow arrays and clarified restore and import behavior in the documentation. Nothing has been pushed yet. |
Validate imported workflows, handle incomplete legacy exports, mask declared step secrets, and defer workflow timer updates until commit. Closes keycloak#50972 Signed-off-by: goutamadwant <workwithgoutam@gmail.com>
b6fdd42 to
aa8e257
Compare
|
The follow-up is now pushed as |
| } | ||
|
|
||
| importOrganizations(rep, newRealm); | ||
| WorkflowExportImportUtils.importWorkflows(session, newRealm, rep); |
| } | ||
|
|
||
| withProvider(session, realm, provider -> { | ||
| workflows.forEach(provider::toModel); |
Summary
Closes #50972
Realm exports currently lose workflow configuration when generic component sanitization removes undeclared properties. This change handles workflow definitions through
WorkflowProvider, preserving their configuration and applying workflow validation during import.workflowsfield.Compatibility
Workflow definitions are restored, not in-progress executions. Step IDs are regenerated on import.
Importing definitions requires the workflows feature. With the feature disabled, offline exports retain stored legacy components, while administrative exports omit them. Partial import into an existing realm is not supported.
Testing
git diff --checkpassed.Full upstream CI has not run.
AI Assistance
AI agents assisted with understanding the codebase, writing tests, and reviewing the solution.