You're right to be concerned about potential double-counting. Here's what's happening:

Token Inference for Spans

Langfuse only infers usage and cost for observations of type generation and embedding, not for regular span types(1). The documentation explicitly states: "Only observations of type generation and embedding can track costs and usage"(1).

However, if the Microsoft Agent Framework is creating observations that Langfuse interprets as generation type (rather than span type), then Langfuse will attempt to infer token counts based on the model parameter if usage details aren't provided(1).

Double-Counting Issues

There are known issues with double-counting in certain scenarios:

1. Parent-Child Generation Observations: If both parent and child observations are of type GENERATION, Langfuse sums tokens from both, causing totals to be roughly doubled(2). The recommended fix is to change the parent observation type from GENERATION to SPAN(2).

2. Cached Tokens: When cached tokens are present (like cache_read_input_tokens), the UI can double-count input tokens by summing all fields containing "input"(3)(4).

Recommendations

To resolve this:

1. Verify observation types: Check whether the Agent Framework is creating span or generation type observations. Only generation types should have token inference applied(1).

2. Explicitly provide usage details: If you want accurate token counts, explicitly ingest the usage details from the Agent Framework's response rather than relying on inference(1).

3. Check for nested generations: Ensure you don't have nested GENERATION type observations that would cause aggregation issues(2).

Could you share a link to a specific trace in Langfuse Cloud so we can examine the observation types and token counting behavior?

📚 Sources:

1. Model Usage & Cost Tracking
2. bug: token statistical error
3. bug: langfuse UI double counts input tokens in the Usage Breakdown hover modal
4. bug: Input tokens counted twice when using cached tokens

Have another question? Just tag @inkeep.

Unfortunately, I cannot access the specific trace link you provided as I don't have direct access to your Langfuse Cloud project.

However, based on the token counting behavior you're describing with the Microsoft Agent Framework, here's what's likely happening:

Token Inference Behavior

Langfuse will attempt to infer token usage when:

1. Usage details are not ingested via the API/SDK
2. The observation is of type generation or embedding
3. A tokenizer is specified for the model(1)

The documentation explicitly states: "Only observations of type generation and embedding can track costs and usage"(1). Regular span type observations should not have token inference applied.

Potential Causes of Double-Counting

If you're seeing double-counted tokens, the most likely scenarios are:

1. The Agent Framework observations are being created as generation type instead of span type, causing Langfuse to infer tokens even though the framework set capture_usage=False

2. Nested generation observations: If both parent and child observations are type generation, Langfuse sums tokens from both levels, effectively doubling the count(1)

How to Verify and Fix

To diagnose this issue, check in the Langfuse UI:

1. Look at the observation type for each Agent Framework span - is it showing as SPAN or GENERATION?
2. Check if there are nested GENERATION type observations in the trace hierarchy
3. Verify whether usage details are being ingested (even if capture_usage=False) or if Langfuse is inferring them

If Agent Framework observations are incorrectly typed as generation, you would need to ensure they're created as span type to prevent automatic token inference(1).

Could you check the observation types in your trace and share what you find?

📚 Sources:

1. Model Usage & Cost Tracking

Have another question? Just tag @inkeep.

I cannot directly access or analyze the JSON file you've attached. However, I can help you understand what to look for based on Langfuse's token counting behavior.

Key Points About Token Inference

Langfuse only infers usage and cost for observations of type generation and embedding—not for regular span types(1). The inference happens when:

• Usage details are not explicitly provided
• The observation has a model parameter
• Langfuse has a tokenizer defined for that model(1)

What to Check in Your Trace JSON

Look for these specific fields in your trace JSON:

1. Observation Types: Check the type field for each observation. If Agent Framework spans are marked as "type": "GENERATION" instead of "type": "SPAN", Langfuse will attempt token inference even though capture_usage=False was set(1).

2. Usage Details: Look for usage or usageDetails fields. If these exist with token counts, they were explicitly ingested. If they're missing but you see token counts in the UI, Langfuse inferred them(1).

3. Nested Generations: Check if you have parent-child relationships where both are type GENERATION. This causes Langfuse to sum tokens from both levels, effectively doubling the count(1).

Expected Behavior

According to the documentation, usage types are summarized as follows: "In the UI, Langfuse summarizes all usage types that include the string input as input usage types, similarly output as output usage types. If no total usage type is ingested, Langfuse sums up all usage type units to a total"(1).

Solution

To prevent double-counting, ensure that Agent Framework observations are created as type span rather than generation. Only the actual LLM call observations should be type generation(1)(2).

If you can share specific observation types and whether usage details are present in the JSON, I can provide more targeted guidance.

📚 Sources:

1. Model Usage & Cost Tracking
2. Instrumentation

Have another question? Just tag @inkeep.

Based on the metadata you've shared, I can now identify the issue. Your invoke_agent span is being sent via OpenTelemetry with the attribute gen_ai.request.model: "gpt-4.1". According to Langfuse's OpenTelemetry property mapping, any span that includes a model attribute is automatically treated as a generation observation type(1).

From the documentation: "Any span that includes an attribute mapped to model is considered an LLM call and thus tracked as a generation observation"(1).

Why Token Inference is Happening

Even though the Microsoft Agent Framework sets capture_usage=False, Langfuse will still attempt to infer token usage because:

1. The span has gen_ai.request.model: "gpt-4.1", which maps to the model property(1)
2. This causes Langfuse to classify it as a generation type observation(1)
3. Langfuse only infers usage and cost for observations of type generation and embedding(2)
4. When usage details aren't provided and a model is specified, Langfuse uses its predefined tokenizers to estimate tokens(2)

The Double-Counting Problem

The double-counting occurs because:

• The Agent Framework's invoke_agent span gets token counts inferred by Langfuse
• Any nested actual LLM calls (like the OpenAI completion) also get their own token counts
• Langfuse sums these together in the total

Solution

To prevent this, you need to ensure the invoke_agent spans do not include model attributes. The Microsoft Agent Framework should only set model attributes on the actual LLM generation spans, not on the orchestration/routing spans.

If you cannot modify the Agent Framework's OpenTelemetry instrumentation, you could alternatively:

1. Manually provide usage details with zero values for the invoke_agent spans using the gen_ai.usage.* attributes(1):

   • gen_ai.usage.input_tokens: 0
   • gen_ai.usage.output_tokens: 0
   • gen_ai.usage.total_tokens: 0

2. Use explicit observation type by setting langfuse.observation.type: "span" to override the automatic generation classification(1)

The key issue is that the presence of gen_ai.request.model automatically triggers generation-type behavior in Langfuse, regardless of the capture_usage=False setting in the Agent Framework.

(2), (1)

📚 Sources:

1. LLM Observability via OpenTelemetry
2. Model Usage & Cost Tracking

Have another question? Just tag @inkeep.