Skip to content

Do not mark cancelled subscription spans as errors - #896

Open
cquiroz wants to merge 1 commit into
mainfrom
otel4s-subscribe-cancel-span
Open

cquiroz wants to merge 1 commit into
mainfrom
otel4s-subscribe-cancel-span

Conversation

@cquiroz

@cquiroz cquiroz commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

We see many trace errors in subscriptions when one is cancelled.

With this PR we avoid that. The subscribe span now marks only real failures as errors and keeps the exception message. Queries and mutations are unchanged.

@mergify mergify Bot added the otel4s label Nov 18, 2026
@mergify

mergify Bot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue). If it does not match the queue conditions yet, it enters the queue once it does.

  • Queue this pull request

operationName,
descriptor,
Otel4sMiddleware.subscriptionFinalization
).build.resource

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MEDIUM - Preexisting] .build.resource doesn't make the subscription span current. The span only becomes current inside res.trace, as the otel4s SpanOps.resource docs say. So T.propagate(Map.empty) (line 248), additionalAttributesF and wrapped.subscribeInternal all run in the caller's span. The traceparent sent in extensions names the caller's span instead of clue-subscribe-*, so the server's spans show up as siblings of the subscription span instead of its children. requestInternal is fine because it uses .use.

Suggested fix: Resource.eval(res.trace(T.propagate(Map.empty))) and wrapped.subscribeInternal(...).mapK(res.trace). This could go in a separate PR.

(Comment by Claude)

started.get >> fiber.cancel
assertIO(subscribeStatus(cancel), StatusData.Unset)

test("a failed subscription marks the span as an error with the failure message"):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW] This test never makes the subscription fail. The error is raised in the use body and the stream is never pulled, so the Errored branch of the stream's onFinalizeCase (Otel4sMiddleware.scala:266-268) is never run. You could delete that branch and both tests would still pass.

Suggestion: add a case where the stub returns Stream.raiseError and the test drains it inside use. Then rename this one to describe a consumer failure.

(Comment by Claude)

(exitCase match
case Resource.ExitCase.Errored(e) =>
span.setStatus(StatusCode.Error, Option(e.getMessage).getOrElse(e.toString))
span.setStatus(StatusCode.Error, Otel4sMiddleware.errorMessage(e))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW - Preexisting] If the consumer recovers from a stream error inside use (e.g. handleErrorWith(_ => Stream.empty)), this branch sets the Error status, but use ends normally. The finalization strategy then does nothing, so the span ends as Error(msg) with no exception event.

Suggestion: call span.recordException(e) here too.

(Comment by Claude)

Option(e.getMessage).getOrElse(e.toString)

// Cancelling is how a subscription normally ends, so only real failures mark the span as errored.
private[otel4s] val subscriptionFinalization: SpanFinalizer.Strategy =

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW] Not marking cancellation as an error makes sense. But if the subscription is cancelled before the stream is pulled, the cancel is no longer recorded anywhere. onFinalizeCase never runs, so clue.exitCase isn't set, and the span looks the same as a normal end.

Suggestion: record it as a plain attribute, not as an error status:

case Resource.ExitCase.Canceled =>
  SpanFinalizer.addAttributes(Attribute("clue.exitCase", "Canceled"))

When both finalizers run they write the same value, so nothing clashes.

(Comment by Claude)

@@ -246,7 +265,7 @@ class Otel4sStreamingClient[F[_]: {Concurrent, Tracer as T}, S](
span.addAttribute(Attribute("clue.exitCase", exitCase.toOutcome.toString)) *>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW - Preexisting] exitCase.toOutcome.toString gives values that are hard to filter on. A failure gives Errored(java.lang.RuntimeException: boom), and a normal end prints the wrapped effect, something like Succeeded(IO(...)). Plain Succeeded / Errored / Canceled values would be easier to query. This would go well with the Canceled case suggested on subscriptionFinalization.

(Comment by Claude)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants