Skip to content

slegarraga/llm-portability-demo

llm-portability-demo

CI OpenSSF Scorecard license offline demo

An offline, one-command proof that the five provider-portability packages can compose into a useful agent flow:

  1. Recover a structured agent plan from model-shaped prose.
  2. Generate provider-native tool schemas from one JSON Schema.
  3. Parse an OpenAI-compatible streaming tool call.
  4. Execute a deterministic local tool and append the tool result.
  5. Normalize a provider failure into a routing decision.
  6. Convert the same OpenAI-hub conversation into Anthropic and Gemini request bodies.

No network call is made unless you explicitly opt into live mode.

Quick start

npm install
npm start

Run the deterministic smoke check:

npm run check

npm run check syntax-checks the demo and checker, executes node demo.mjs offline, compares the transcript with docs/sample-output.txt, and asserts the concrete handoffs between packages. CI runs the same command.

Expected output

The full transcript is committed at docs/sample-output.txt. The important shape is:

1) json-from-llm  recover an agent plan from reasoning/prose output
2) tool-schema    one JSON Schema -> OpenAI, Anthropic, Gemini, MCP
3) llm-sse        parse the stream and collect the tool call
4) offline tool   execute get_weather with deterministic fixture data
5) llm-errors     primary provider failure becomes a routing decision
6) llm-messages   same OpenAI-hub history, provider-native fallback bodies

Proof matrix
   json-from-llm -> plan city=Santiago units=metric fallback=anthropic
   tool-schema   -> OpenAI tool=get_weather, MCP output fields=5
   llm-sse       -> assistant tool_call=get_weather args=city,units
   offline tool  -> fixture result=Santiago/18.4 metric clear
   llm-errors    -> openai/rate_limit retryAfter=30000ms fallback=anthropic
   llm-messages  -> Anthropic messages=3, Gemini contents=3

Proof: one offline agent plan, one tool schema, one streamed tool call, one error policy, portable provider bodies.

Architecture

The demo keeps OpenAI Chat Completions messages as the canonical hub because that is the common shape many OpenAI-compatible providers already speak.

model-like prose
  -> json-from-llm extracts { city, units, fallbackProvider, maxRetryMs }
  -> tool-schema emits OpenAI, Anthropic, Gemini and MCP tool schemas
  -> llm-sse parses an OpenAI-compatible SSE stream into a tool-call message
  -> offline get_weather fixture returns deterministic tool output
  -> llm-errors normalizes a 429 + Retry-After into a fallback decision
  -> llm-messages ports the same hub history to Anthropic and Gemini bodies

This proves the packages share practical data boundaries without depending on each other at runtime. The demo has only the five package dependencies and can be run in CI, on an airplane, or inside a fresh clone after npm install.

Package roles

  • json-from-llm recovers JSON from reasoning, markdown or prose output.
  • tool-schema turns one JSON Schema into valid OpenAI, Anthropic, Gemini and MCP tools, including MCP outputSchema.
  • llm-sse parses streaming provider responses into unified events and collects an assistant message.
  • llm-errors normalizes provider failures so agents can retry, fail, or fall back intentionally.
  • llm-messages ports the identical conversation to another provider request shape.

Live provider mode

Live mode is opt-in and only runs when LLM_DEMO_LIVE=1 is set. A provider API key by itself is not enough, which keeps CI and local demos cost-safe by default.

LLM_DEMO_LIVE=1 \
LLM_DEMO_API_KEY="$OPENAI_API_KEY" \
LLM_DEMO_MODEL="gpt-4o-mini" \
npm start

For OpenAI-compatible providers, point the demo at a compatible /v1 base URL:

LLM_DEMO_LIVE=1 \
LLM_DEMO_BASE_URL="https://openrouter.ai/api/v1" \
LLM_DEMO_API_KEY="$OPENROUTER_API_KEY" \
LLM_DEMO_MODEL="openai/gpt-4o-mini" \
npm start

Live-mode rules:

  • LLM_DEMO_LIVE=1 is required to make any network call.
  • LLM_DEMO_API_KEY and LLM_DEMO_MODEL are required in live mode.
  • LLM_DEMO_BASE_URL is optional and defaults to https://api.openai.com/v1.
  • The demo never prints API keys, request headers or error response bodies.
  • Live calls may incur provider costs; use a cheap model and a low-limit key.

Suite status

Package npm Downloads CI
tool-schema npm downloads CI
llm-sse npm downloads CI
llm-errors npm downloads CI
llm-messages npm downloads CI
json-from-llm npm downloads CI

Download badges use the npm downloads range API over the last 30 complete days because the npm monthly point endpoint can lag for newly published packages.

More docs

Releases

No releases published

Packages

 
 
 

Contributors