Export traces

Overview

LiveKit Agents instruments each session with OpenTelemetry traces: the same spans that power Agent insights in LiveKit Cloud. Set a tracer provider to export these spans to any OpenTelemetry-compatible backend.

The following example sends spans to Langfuse , an open-source LLM observability platform. The same approach works for any backend that accepts traces over the OpenTelemetry Protocol (OTLP). To learn more, see Other backends.

Set environment variables

Create an API key pair in your Langfuse project settings, then add the following to your agent's .env.local file:

• LANGFUSE_PUBLIC_KEY: The public key for your Langfuse project.
• LANGFUSE_SECRET_KEY: The secret key for your Langfuse project.
• LANGFUSE_BASE_URL: The URL for your Langfuse instance, such as https://cloud.langfuse.com (EU) or https://us.cloud.langfuse.com (United States).

The example script reads these variables to build the OTLP endpoint and authentication headers that the exporter sends to Langfuse. To export to a different backend, set those values directly instead. See Other backends.

Trace a complete agent

Both examples send the x-langfuse-ingestion-version header to opt into Langfuse's realtime ingestion. Without it, spans can take up to 10 minutes to appear.

import base64
import os

from dotenv import load_dotenv
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.util.types import AttributeValue

from livekit.agents import (
    Agent,
    AgentServer,
    AgentSession,
    JobContext,
    cli,
    inference,
)
from livekit.agents.telemetry import set_tracer_provider

load_dotenv(".env.local")


def setup_langfuse(metadata: dict[str, AttributeValue] | None = None) -> TracerProvider:
    from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
    from opentelemetry.sdk.trace.export import BatchSpanProcessor

    public_key = os.environ.get("LANGFUSE_PUBLIC_KEY")
    secret_key = os.environ.get("LANGFUSE_SECRET_KEY")
    base_url = os.environ.get("LANGFUSE_BASE_URL")

    if not public_key or not secret_key or not base_url:
        raise ValueError("LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, and LANGFUSE_BASE_URL must be set")

    langfuse_auth = base64.b64encode(f"{public_key}:{secret_key}".encode()).decode()
    os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = f"{base_url.rstrip('/')}/api/public/otel"
    os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = (
        f"Authorization=Basic {langfuse_auth},x-langfuse-ingestion-version=4"
    )

    trace_provider = TracerProvider()
    trace_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
    set_tracer_provider(trace_provider, metadata=metadata)
    return trace_provider


class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions="You are a helpful voice AI assistant.",
            llm=inference.LLM(model="openai/gpt-5.2-chat-latest"),
        )


server = AgentServer()


@server.rtc_session(agent_name="my-agent")
async def entrypoint(ctx: JobContext):
    # Route spans to Langfuse before the session starts.
    trace_provider = setup_langfuse(metadata={"langfuse.session.id": ctx.room.name})

    # Flush any remaining spans before the process exits.
    async def flush_trace():
        trace_provider.force_flush()

    ctx.add_shutdown_callback(flush_trace)

    session = AgentSession(
        stt=inference.STT(model="deepgram/nova-3", language="multi"),

        tts=inference.TTS(model="inworld/inworld-tts-2"),

        preemptive_generation=True,
    )

    await session.start(agent=Assistant(), room=ctx.room)
    await ctx.connect()


if __name__ == "__main__":
    cli.run_app(server)

pnpm add @opentelemetry/api @opentelemetry/sdk-trace-node @opentelemetry/exporter-trace-otlp-http

import {
  type JobContext,
  ServerOptions,
  cli,
  defineAgent,
  inference,
  telemetry,
  voice,
} from '@livekit/agents';
import { type Attributes } from '@opentelemetry/api';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { BatchSpanProcessor, NodeTracerProvider } from '@opentelemetry/sdk-trace-node';
import dotenv from 'dotenv';
import { fileURLToPath } from 'node:url';

dotenv.config({ path: '.env.local' });

function setupLangfuse(metadata?: Attributes): NodeTracerProvider {
  const publicKey = process.env.LANGFUSE_PUBLIC_KEY;
  const secretKey = process.env.LANGFUSE_SECRET_KEY;
  const baseUrl = process.env.LANGFUSE_BASE_URL;

  if (!publicKey || !secretKey || !baseUrl) {
    throw new Error('LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, and LANGFUSE_BASE_URL must be set');
  }

  const auth = Buffer.from(`${publicKey}:${secretKey}`).toString('base64');
  const traceExporter = new OTLPTraceExporter({
    url: `${baseUrl.replace(/\/$/, '')}/api/public/otel/v1/traces`,
    headers: { Authorization: `Basic ${auth}`, 'x-langfuse-ingestion-version': '4' },
  });

  // A provider takes its span processors at construction. Include a FanoutSpanProcessor and
  // hand its `add` method to setTracerProvider so the framework can attach the processor that
  // applies `metadata` to every span.
  const fanout = new telemetry.FanoutSpanProcessor();
  const traceProvider = new NodeTracerProvider({
    spanProcessors: [new BatchSpanProcessor(traceExporter), fanout],
  });

  traceProvider.register();
  telemetry.setTracerProvider(traceProvider, {
    metadata,
    registerSpanProcessor: (processor) => fanout.add(processor),
  });
  return traceProvider;
}

export default defineAgent({
  entry: async (ctx: JobContext) => {
    // Route spans to Langfuse before the session starts.
    const traceProvider = setupLangfuse({ 'langfuse.session.id': ctx.room.name });

    // Flush any remaining spans before the process exits.
    ctx.addShutdownCallback(async () => {
      await traceProvider.shutdown();
    });

    const session = new voice.AgentSession({
      stt: new inference.STT({ model: 'deepgram/nova-3', language: 'multi' }),
      llm: new inference.LLM({ model: 'openai/gpt-5.2-chat-latest' }),
      tts: new inference.TTS({
        model: 'inworld/inworld-tts-2',
        voice: 'Ashley',
      }),
    });

    await session.start({
      agent: voice.Agent.create({ instructions: 'You are a helpful voice AI assistant.' }),
      room: ctx.room,
    });

    await ctx.connect();
  },
});

cli.runApp(new ServerOptions({ agent: fileURLToPath(import.meta.url), agentName: 'my-agent' }));

Span attributes

Agent spans carry attributes from two namespaces. The lk.* namespace holds LiveKit-specific data such as speech IDs, turn timings, and serialized metrics. The gen_ai.* namespace follows the OpenTelemetry GenAI semantic conventions, which most observability backends use for model usage and cost reporting.

Content attributes

Attributes that carry conversation content, tool payloads, or participant data are named with an lk.pii. prefix. The prefix lets LiveKit Cloud strip them when PII redaction is turned on, and it's applied in the SDK regardless of whether you use LiveKit Cloud. To keep these attributes out of your own backend, see Strip PII from exported traces.

Agents 1.7.0 added the prefix to all of these attributes. Python and Node.js use the same 12 names:

The rename is silent for downstream consumers. A Langfuse view, Datadog monitor, or custom query that uses an old name returns no matches after you upgrade rather than raising an error. Update these queries as part of the upgrade. Attributes that don't contain content, such as lk.speech_id, lk.job_id, and the gen_ai.usage.* token counts, keep their existing names.

Agents 1.7.0 also moves roughly 50 structured log fields to the same naming convention. Most are plugin-level fields that contain transcripts, tool payloads, or raw provider messages. The affected fields differ by SDK:

• Python: keys in the extra dict on a log record, such as chat_ctx → lk.pii.chat_ctx, arguments → lk.pii.arguments, and transcript → lk.pii.transcript.
• Node.js: Pino child-logger fields, including roomName → lk.pii.room_name and participant or participantIdentity → lk.pii.participant_identity. These two renames are specific to Node.js.

If you consume agent logs through a log drain, update any filters or dashboards that use the old field names.

To tag your own attributes and log fields for redaction by LiveKit Cloud, see Tag your own attributes.

Cached input tokens

LLM request spans report token usage with the gen_ai.usage.* attributes from the OpenTelemetry GenAI semantic conventions . The gen_ai.usage.cache_read.input_tokens attribute reports the input tokens served from the prompt cache.

Strip PII from exported traces

By default, exporters receive the full span, including conversation content, tool payloads, and participant data. A GenAI-aware backend can only render a conversation it receives. To withhold that content from your own backend, pass allow_pii=False (Python) or allowPii: false (Node.js) when you set the tracer provider.

set_tracer_provider(trace_provider, metadata=metadata, allow_pii=False)

telemetry.setTracerProvider(traceProvider, {
  metadata,
  registerSpanProcessor: (processor) => fanout.add(processor),
  allowPii: false,
});

If you don't call set_tracer_provider and the SDK adopts the global OpenTelemetry provider instead, set the LIVEKIT_TELEMETRY_ALLOW_PII environment variable to 0. An explicit argument takes precedence over the environment variable.

This option doesn't change what LiveKit Cloud receives. The SDK's own exporter for Agent insights still gets the full span, and your project's PII redaction setting governs what LiveKit Cloud stores.

What the SDK removes

With allow_pii=False, the SDK strips the following from each span before any exporter on the provider sees it:

• Attributes with a pii segment in the key, including all content attributes and any attributes you tag yourself.
• GenAI content attributes: gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions, gen_ai.tool.call.arguments, gen_ai.tool.call.result, gen_ai.tool.description, gen_ai.tool.definitions, and gen_ai.prompt.variable.*.
• GenAI message events, such as gen_ai.choice and gen_ai.user.message.
• Exception details. exception.message and the span's error status become exception details redacted, and exception.stacktrace is dropped.

Attributes that don't carry content remain, including gen_ai.usage.* token counts, model names, provider names, durations, and lk.speech_id. The SDK drops attributes whole. It doesn't scan or mask values.

Turn off content capture

To remove gen_ai.* content attributes from all spans, including those sent to LiveKit Cloud, set the standard OpenTelemetry environment variable OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT to false. You can also call telemetry.gen_ai.set_capture_content(False) (Python) or telemetry.genAI.setCaptureContent(false) (Node.js) before the session starts. This affects only the gen_ai.* content attributes. The lk.pii.* attributes are still recorded, so use allow_pii to strip those.

Other backends

The preceding pattern works for any backend that accepts OpenTelemetry traces over OTLP. To export elsewhere, point the exporter at the OTLP endpoint for that backend and set the authentication it requires:

• OTEL_EXPORTER_OTLP_ENDPOINT: The OTLP HTTP endpoint for the backend.
• OTEL_EXPORTER_OTLP_HEADERS: Any authentication headers the backend requires, such as an API key.

The rest of the agent stays the same: build a tracer provider, add a batch span processor with an OTLP exporter, and pass the provider to set_tracer_provider (Python) or telemetry.setTracerProvider (Node.js) before the session starts.

In Node.js you can also pass url and headers to the exporter instead of setting environment variables, as the preceding example does. The url option is used as-is, while OTEL_EXPORTER_OTLP_ENDPOINT has /v1/traces appended to it.

Additional resources