Skip to content

Close all streams in the Keycloak production code - #53852

Merged
stianst merged 5 commits into
keycloak:mainfrom
ahus1:is-53845-ensure-closed-streams
Nov 13, 2026
Merged

stianst merged 5 commits into
keycloak:mainfrom
ahus1:is-53845-ensure-closed-streams

Conversation

@ahus1

@ahus1 ahus1 commented Oct 12, 2026

Copy link
Copy Markdown
Member

Closes #53845

@lhanusov

lhanusov commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

@ahus1 As Tomas writes, please, don't introduce 3rd party annotations into the code, also is it possible to remove SuppressWorning, related to checker only? This would also remove additional checker pom dependencies.

@lhanusov

Copy link
Copy Markdown
Contributor

@ahus1 did you consider codeql, which is already part of the CI?

@lhanusov

Copy link
Copy Markdown
Contributor

@ahus1 Besides that, static analysis makes sense, but only if managed by external config, not touching the codebase itself. So, partially agree with your proposal, thanks for this PR. 👍

Comment thread services/pom.xml Outdated
<groupId>org.checkerframework</groupId>
<artifactId>checker-qual</artifactId>
<version>${checkerframework.version}</version>
<scope>provided</scope>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect in all pom.xml files.
provided means the dependency will be available at runtime, provided by a container or the JDK. That is not the case.

As the annotation is present at runtime, I would add the dependency to the distribution. I'm aware that the JVM loads the annotation class only if reflection is used, but I think it's better to be safe than to get a NoClassDefFoundError.

In any case, <scope>provided</scope> is not right. You probably want to make it optional.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore me, others already commented the same thing 😅

@ahus1

ahus1 commented Oct 20, 2026

Copy link
Copy Markdown
Member Author

@lhanusov

please, don't introduce 3rd party annotations into the code, also is it possible to remove SuppressWorning, related to checker only? This would also remove additional checker pom dependencies

The jdk.astub is only the workaround for those classes that we do not own, and it is quite brittle. Given that tradeoff, I would say we should not do it.

did you consider codeql, which is already part of the CI?

The tool should be self-contained. So far CodeQL didn't warn us about those problems, so we would need to write our own rules. Is that what you are suggeting?

but only if managed by external config, not touching the codebase itself.

In an ideal world yes. Unless you show me a tool that can do this, this is IMHO as good as it gets.

@ahus1

ahus1 commented Oct 20, 2026

Copy link
Copy Markdown
Member Author

I asked Copilot: Could this be done with CodeQL instead of the Checker Framework?

And this is the answer:

Partly, but not with the same strength.

What CodeQL offers

  • CodeQL ships java/input-resource-leak and java/output-resource-leak.
    They flag streams, readers and writers that are not closed on all paths.
    They are not in the default suite, but security-and-quality includes them.
  • The repository already runs codeql-analysis.yml for Java, so enabling them is cheap.
    It needs queries: security-and-quality on the init step, or a config file as for JS/TS.
  • There would be no JDK 21 pin, no -Pcheckerframework profile, no checker-qual dependencies, no per-module compiler overrides and no jdk.astub.

What CodeQL does not do

  • The ResourceLeakChecker is interprocedural and typed.
    It uses @Owning, @MustCall and @NotOwning, so it can verify that a stream returned from a method is closed by the caller, or that a field is closed in close().
    This PR relies on that in places like HttpClientProvider, SafeInputStream, SimpleHttpResponse, KeycloakSession and DefaultKeycloakSession.
  • The stock CodeQL queries are mostly local.
    They tend to treat a stream that is returned, stored in a field or passed to another method as handled.
    They would likely miss many of the leaks fixed here, and can produce false positives on wrappers.
  • Matching the Checker Framework's strictness would need a custom QL query plus an ownership convention, which is real work and less precise.
  • CodeQL findings go to code scanning and do not fail the PR build unless gating is added.

Overrides and false positives with CodeQL

  • Alerts can be dismissed in the GitHub code scanning UI as "False positive", "Used in tests" or "Won't fix", with a comment.
    The dismissal is stored in GitHub and is not visible in the source code.
  • Java has no inline suppression by default.
    The alert-suppression query enables comments like // codeql[java/input-resource-leak] on the affected line.
    It needs to be added via the queries: entry of the config file.
  • A config file (codeql-config-*.yml) can use paths-ignore: to exclude directories, or query-filters: to exclude queries by id, tags or precision.
    This is coarse, and a single method can't be excluded this way.
  • A helper that really closes its argument can be modelled with a data extension (YAML) or a custom QL library.
    This is the cleanest long-term fix, but it takes effort, and it would also be needed to approximate @Owning.
  • The Checker Framework offers @SuppressWarnings("resource"), @Owning and @NotOwning.
    These are precise, attached to the exact element and visible in code review.
    CodeQL's UI dismissals and the extra suppression query are weaker in this respect.

Recommendation

  • Treat CodeQL as a cheap complement, not a replacement.
    Enabling the two stock queries gives some ongoing protection without the build complexity.
  • If the goal is a hard guarantee that all streams are closed, keep the Checker Framework.

I (Copilot) have not run the CodeQL queries against this branch, so this is based on how the queries are designed rather than measured results.

@ahus1
ahus1 force-pushed the is-53845-ensure-closed-streams branch 2 times, most recently from e58566c to 3759780 Compare October 20, 2026 12:45
@ahus1

ahus1 commented Oct 20, 2026

Copy link
Copy Markdown
Member Author

Picking up an idea from @ruchikajha95 to only focus on production code, and not on Test classes in this first PR.

@pruivo shared a review with Claude, and I've incorporated those changes.

There is in building.md now also some docs on how to use the annotations correctly, and why some classes are excluded.

Please give it another view, and let me know if we should continue with this tool to check for closed streams (and have additional follow-up PRs to cover more production code and also tests), or rather look for a different tool.

@ruchikajha95

ruchikajha95 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Picking up an idea from @ruchikajha95 to only focus on production code, and not on Test classes in this first PR.

@pruivo shared a review with Claude, and I've incorporated those changes.

There is in building.md now also some docs on how to use the annotations correctly, and why some classes are excluded.

Please give it another view, and let me know if we should continue with this tool to check for closed streams (and have additional follow-up PRs to cover more production code and also tests), or rather look for a different tool.

Thanks for the updates, @ahus1! Really appreciate you incorporating the feedback and adding the docs in building.md that should be very helpful . I personally feel this is a great addition. I personally feel We should move forward with this approach.
Looking forward to the follow-up PRs. Happy to help!

Comment thread docs/building.md Resolved
stianst

This comment was marked as outdated.

@ahus1
ahus1 marked this pull request as ready for review October 26, 2026 08:47
@ahus1
ahus1 requested review from a team as code owners October 26, 2026 08:47
Copilot AI balanced review requested due to automatic review settings October 26, 2026 08:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-module compiler integration and ownership contracts need maintainer review backed by a successful clean JDK 21 checker build.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@ahus1
ahus1 dismissed tkyjovsk’s stale review October 26, 2026 12:01

resolved in chat.

protected static final Logger log = Logger.getLogger(PushDeliveryService.class);

private final HttpClient httpClient;
private final CloseableHttpClient httpClient;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this client also be closed at some point?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This client is retrieved from session.getProvider(HttpClientProvider.class).getHttpClient();, and it therefore must not be closed here. It is a shared client, and it will eventually be closed by the provider.

As the JavaDoc states:

Closing the HttpClient instance is responsibility of this provider. However, the objects created via the returned HttpClient need to be closed properly by the code that instantiated them.

As a follow-up, the interface will return a HttpClient, not a CloseableHttpClient. But that's a breaking change for the API, though it should have been like this all the time.

I slightly tune the annotations that this can stay a HttpClient, see my lastest push.

IMHO this resolved this.

Copilot AI balanced review requested due to automatic review settings November 2, 2026 10:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The compiler-profile changes affect reactor-wide annotation processing and module-specific processors, but the required dependency-backed clean JDK 21 reactor build is unavailable in this review environment.

No open issues 🎉

Review effort: Balanced

@ahus1
ahus1 requested a review from stianst November 2, 2026 10:38
@ahus1
ahus1 force-pushed the is-53845-ensure-closed-streams branch from 3dfdcd6 to 54bdbab Compare November 2, 2026 13:19
Copilot AI balanced review requested due to automatic review settings November 2, 2026 13:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the resource-cleanup and static-checking changes.

No open issues 🎉

Review effort: Balanced

@stianst stianst self-assigned this Nov 10, 2026
Comment thread docs/static-code-analysis.md Outdated
### Fixing findings

* Prefer try-with-resources for resources that are closed in the same method.
* Use `@Owning` on a parameter or field when the callee or the containing object becomes responsible for closing the resource.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This explanation doesn't make any sense to me; callee would mean the caller is responsible for closing the resource, while containing object means the called object is responsible.

Might be better to explain @Owning and @NotOwning` together, with the difference between it being used on an field or parameter (in this case @owning means the called object owns it), while on a method return it means the caller owns it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stianst - I've restructured the guide, let me know if this helps. I also updated to the latest version of the Checker framework to remove some of the suppressions.

Comment thread docs/static-code-analysis.md Outdated

* Prefer try-with-resources for resources that are closed in the same method.
* Use `@Owning` on a parameter or field when the callee or the containing object becomes responsible for closing the resource.
See [`SafeInputStream`](../server-spi-private/src/main/java/org/keycloak/connections/httpclient/SafeInputStream.java).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove the See links. For one it is confusing as See suggests it's more documentation/explanation, not an example. Link to the checker framework docs instead perhaps? Or, provide some simple examples in the readme?

Comment thread docs/static-code-analysis.md Outdated
See [`SafeInputStream`](../server-spi-private/src/main/java/org/keycloak/connections/httpclient/SafeInputStream.java).
* Use `@NotOwning` on a method that returns a resource managed by someone else.
See [`HttpClientProvider`](../server-spi-private/src/main/java/org/keycloak/connections/httpclient/HttpClientProvider.java).
* Use `@MustCallAlias` for wrappers, where closing the wrapper closes the wrapped resource, and `@EnsuresCalledMethods` on the `close()` method that closes owned fields.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above really, this explanation doesn't help me, just confuses me.

Comment thread docs/static-code-analysis.md Outdated
* Use `@NotOwning` on a method that returns a resource managed by someone else.
See [`HttpClientProvider`](../server-spi-private/src/main/java/org/keycloak/connections/httpclient/HttpClientProvider.java).
* Use `@MustCallAlias` for wrappers, where closing the wrapper closes the wrapped resource, and `@EnsuresCalledMethods` on the `close()` method that closes owned fields.
* Contracts for third-party APIs are declared in `.mvn/checkerframework/jdk.astub`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about @MustCall that is an important one and should be explained?

Comment thread docs/static-code-analysis.md Resolved
Closes keycloak#53845

Signed-off-by: Alexander Schwartz <alexander.schwartz@ibm.com>
Signed-off-by: Alexander Schwartz <alexander.schwartz@ibm.com>
Signed-off-by: Alexander Schwartz <alexander.schwartz@ibm.com>
Signed-off-by: Alexander Schwartz <alexander.schwartz@ibm.com>
Signed-off-by: Alexander Schwartz <alexander.schwartz@ibm.com>
@ahus1
ahus1 force-pushed the is-53845-ensure-closed-streams branch from 54bdbab to 028b7ad Compare November 10, 2026 14:37
Copilot AI balanced review requested due to automatic review settings November 10, 2026 14:37
@ahus1
ahus1 requested a review from stianst November 10, 2026 14:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the resource-cleanup fixes and static-analysis additions.

No open issues 🎉

Review effort: Balanced

@stianst
stianst merged commit a9aad4c into keycloak:main Nov 13, 2026
108 of 110 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ensure all streams are closed

7 participants