NodeLLM Monitor 🛰️

Production-grade observability for LLM applications in Node.js.

"Datadog for LLMs — built for Node.js"

NodeLLM Monitor provides infrastructure-first monitoring for AI applications. It captures requests, tool calls, costs, performance metrics, and traces across providers and SDKs — with a standalone real-time dashboard.

Modern AI apps fail silently. You need visibility into:

• Request latency — How long are LLM calls taking?
• Token usage & cost — What's the actual spend per request?
• Tool execution — Which tools are being called, and are they succeeding?
• Streaming behavior — What's the time-to-first-token?
• Error rates — Which requests are failing and why?
• Provider reliability — Is OpenAI slower than Anthropic today?

NodeLLM Monitor gives you deep observability without vendor lock-in.

• Teams running LLMs in production — You need visibility, not just logs
• Node.js backend engineers building AI features — First-class TypeScript support
• Platforms needing provider-agnostic telemetry — Works with any LLM provider
• Apps using NodeLLM, Vercel AI SDK, LangChain, or OpenTelemetry pipelines

NodeLLM Monitor is designed as a decoupled observability layer:

LLM SDK / Framework
        ↓
NodeLLM Middleware OR OpenTelemetry Bridge
        ↓
Monitoring Engine
        ↓
Storage Adapter (Memory / File / Prisma / Custom)
        ↓
Dashboard + API

It operates in two primary modes:

1. Native Middleware: First-class integration with @node-llm/core
2. OpenTelemetry Bridge: Zero-code instrumentation for Vercel AI SDK, LangChain, or any OTel-compatible library

Package	Version	Description
@node-llm/monitor		Core monitoring engine, storage adapters, and dashboard
@node-llm/monitor-otel		OpenTelemetry bridge for AI observability

pnpm add @node-llm/monitor

If you are using @node-llm/core, adding monitoring is a single line:

import { createLLM } from "@node-llm/core";
import { Monitor } from "@node-llm/monitor";

const monitor = Monitor.memory(); // Or File/Prisma adapter

const llm = createLLM({
  provider: "openai",
  model: "gpt-4o",
  middlewares: [monitor]
});

// All calls via 'llm.ask()' are now automatically tracked!

Zero-code instrumentation for existing OTel-instrumented libraries:

import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
import { NodeLLMSpanProcessor } from "@node-llm/monitor-otel";
import { Monitor } from "@node-llm/monitor";

const monitor = Monitor.memory();
const provider = new NodeTracerProvider();

// Hook the AI-aware SpanProcessor into your OTel pipeline
provider.addSpanProcessor(new NodeLLMSpanProcessor(monitor.getStore()));
provider.register();

import express from "express";
import { Monitor } from "@node-llm/monitor";

const monitor = Monitor.memory();
const app = express();

// Standalone dashboard available at /monitor
app.use(monitor.api({ basePath: "/monitor" }));

app.listen(3000);

Track the pulse of your AI infrastructure:

• Throughput: Request volume and error rates.
• Cost: Automatic token counting and USD cost calculation for major providers.
• Performance: Latency tracking, Time-to-First-Token (TTFT), and tokens/sec.

Inspect the full lifecycle of every AI request:

• Tool Calls: See exactly what tools were called, their arguments, and results.
• Streaming: Visualize the progression of streamed responses.
• Content: Optional request/response content capture for debugging.

Production-safe by default:

• Zero-Storage Content: Content capture is disabled by default.
• Automated Scrubbing: Built-in identifiers for PII (emails, keys, etc.) that mask sensitive data before it hits your database.

NodeLLM Monitor works seamlessly with:

• NodeLLM Core — Native middleware integration
• Vercel AI SDK — Via OpenTelemetry bridge
• LangChain — Via OpenTelemetry bridge
• Any OpenTelemetry instrumented system

This is a pnpm workspace. To get started:

# Install dependencies
pnpm install

# Build all packages
pnpm build

# Run the test suite
pnpm test

Note: If you encounter tsx: command not found when running examples, ensure you've run pnpm install at the root to link workspace binaries.

We provide detailed examples for various scenarios:

Contributions welcome! Please read:

• CONTRIBUTING.md
• CODE_OF_CONDUCT.md

Please report vulnerabilities responsibly via SECURITY.md.

MIT © Shaiju Edakulangara

Built with ❤️ for the Node.js AI community