Agenda​

• Product updates and roadmap progress
• Reliability and security updates
• Open Q&A with the team

How to contribute​

Add your thoughts to this ticket to help us shape the agenda.

Register​

Register here: LiteLLM May Townhall Form

We will hold the townhall from 7:30 AM to 8:30 AM PST on Zoom.

For security, attendance is restricted to corporate emails. If you register with a non-corporate email, we will share the townhall slides and accompanying blog post after the event.

The main benefit of using this is that it will manage:

• Different sandboxes for different teams/contexts
• Session management across pod restarts/upgrades

We built this because we wanted a managed agent solution, but fully self-hosted. We are excited to have it open sourced and available for everyone to use.

Repo: github.com/BerriAI/litellm-agent-platform

Please file an issue if you have any questions or feedback.

We received a report through our bug bounty program regarding a SQL injection vulnerability in LiteLLM Proxy's API key verification path, tracked as CVE-2026-42208.

The issue was reviewed by our team, fixed in a stable release, and then published as a GitHub Security Advisory.

• Affected versions: v1.81.16 through v1.83.6
• Fixed versions: v1.83.7 and later
• Recommended version: v1.83.10-stable

Stable release: https://github.com/BerriAI/litellm/releases/tag/v1.83.10-stable

Advisory: https://github.com/BerriAI/litellm/security/advisories/GHSA-r75f-5x8p-qvmc

TLDR;​

• This issue was reported through LiteLLM's bug bounty program.
• We fixed the issue in a stable release before publishing the GitHub Security Advisory.
• LiteLLM Proxy versions v1.81.16 through v1.83.6 are affected.
• The fix is available in v1.83.7 and later.
• We recommend upgrading to v1.83.10-stable.
• If your proxy was reachable from an untrusted network while running an affected version, we recommend reviewing Postgres query history using the helper query below.

What was the issue?​

LiteLLM Proxy validates incoming requests by checking the Authorization: Bearer header during API key verification.

In affected versions, an unauthenticated request with a crafted Authorization: Bearer header could, under certain conditions, reach a vulnerable database query path.

This could potentially result in unintended database access. Practical impact depends on deployment configuration, network exposure, database permissions, and stored data.

Our security process​

This issue was reported through our bug bounty program. Our team reviewed the report, patched the vulnerable path, validated the fix, and released a stable build before publishing the GitHub Security Advisory.

We follow this process so users have a clear remediation path available at the time an advisory is published.

What you should do​

1. Upgrade to v1.83.10-stable​

We recommend upgrading to the latest stable release:

https://github.com/BerriAI/litellm/releases/tag/v1.83.10-stable

If you are unable to upgrade directly to v1.83.10-stable, upgrade to any version v1.83.7 or later.

2. Review Postgres query history if applicable​

If your LiteLLM Proxy was reachable from an untrusted network while running an affected version, we recommend reviewing your Postgres query history using this helper query:

https://gist.github.com/ishaan-berri/6f31e56e878338eb4c01990bd08378ab

If the query returns results you'd like us to review, send them over and we can help triage.

Continuing to invest in security​

We will continue investing in our bug bounty program and coordinated disclosure process so issues can be identified, fixed, and communicated responsibly.

If you find a security issue in LiteLLM, please report it through our GitHub Security Advisory process or our bug bounty program.

1. The -stable and -nightly suffixes aren't standard.

Versions like v1.83.3-stable and v1.83.0-nightly don't match PEP 440 (PyPI) or SemVer 2.0 (Docker / Helm) conventions. Users expecting standard version strings get confused, and tooling that classifies versions has to special-case the suffix.

2. Weekly releases were bumping PATCH, leaving no room for actual hotfixes.

Under the old model, each scheduled weekly release bumped the PATCH number: 1.83.0 -> 1.83.1 -> 1.83.2 -> 1.83.3. When a real hotfix was needed for 1.83.3, the next PATCH (1.83.4) was already reserved for the following week's release. The workaround on Docker was v1.83.3-stable.patch.1 - but PyPI doesn't accept that syntax, so a hotfix that needed both a Docker image and a Python wheel had no clean way to ship.

What's new​

Starting with 1.84.0:

• Drop the suffix. Stable releases are plain PEP 440 / SemVer 2.0: 1.84.0. Pre-releases use the standard PEP 440 (1.84.0rc1, 1.84.0.dev42) and SemVer (1.84.0-rc.1, 1.84.0-dev.42) shapes for PyPI and Docker respectively.
• MINOR bumps weekly. Each scheduled stable bumps the MINOR component: 1.84.0 -> 1.85.0 -> 1.86.0.
• PATCH is reserved for hotfixes. When 1.84.0 needs a fix, it becomes 1.84.1. Cleanly installs everywhere - pip install litellm==1.84.1, docker pull ghcr.io/berriai/litellm:1.84.1.

Side-by-side​

On Docker, every channel is published in both bare (1.84.0) and v-prefixed (v1.84.0) form going forward — both resolve to the same image digest, so existing pins that include the v prefix keep working. On PyPI, every channel uses the bare PEP 440 form (1.84.0, never v1.84.0).

The hotfix row is the meaningful one. Under the old scheme there was no PyPI publication for v1.83.3-stable.patch.1. Under the new scheme, hotfixes ship to both registries and PyPI as a normal release.

Backwards compatibility​

Releases that already shipped with the old naming - v1.83.x-stable, v1.83.x-stable.patch.N, and existing 1.83.x PyPI versions - stay on the registries and PyPI forever. Anything you currently pin to keeps working. The new naming applies to new releases starting 1.84.0.

If a maintenance patch is needed on a pre-cutover release line (e.g. a fix on 1.83.x while 1.84.x is current), that patch may continue to use the old naming for consistency within the line - release notes will call out which format was used. Long-term, all new releases move to the new naming.

A few things worth knowing​

• The v prefix is optional on Docker tags. Every Docker tag going forward is published in both bare and v-prefixed form — ghcr.io/berriai/litellm:1.84.0 and ghcr.io/berriai/litellm:v1.84.0 resolve to the same image (same sha256 digest), and the same applies to release-candidate and dev/nightly tags. Existing pins that include the v prefix keep working without change. PyPI versions remain the bare PEP 440 form: pip install litellm==1.84.0 (not ==v1.84.0).
• litellm-dev - there's a separate litellm-dev PyPI package and *-dev Docker image family for ad-hoc and one-off builds (e.g. testing a fix before it lands in a release). Not for production use. Anything pinned to the standard litellm package or ghcr.io/berriai/litellm:* Docker tags will never accidentally pick up a litellm-dev build.
• :latest Docker tag points to the most recent stable release on each registry, advancing automatically when a new stable ships. For production deployments we still recommend pinning to a content tag (e.g. :1.84.0) so deploys are reproducible.
• Image signing (cosign verify) and verification commands continue to work unchanged with the new tag shapes.

Supported Input Types​

Input Formats​

LiteLLM accepts three input formats for multimodal content:

1. Data URIs – Base64-encoded inline: data:image/png;base64,<encoded_data>
2. GCS URLs – Cloud Storage paths (Vertex AI): gs://bucket/path/to/file.png
3. Gemini File References – Pre-uploaded files (Gemini API): files/abc123

Quick Start​

from litellm import embedding
import os

os.environ["GEMINI_API_KEY"] = "your-api-key"

# Text + Image (base64)
response = embedding(
    model="gemini/gemini-embedding-2",
    input=[
        "The food was delicious and the waiter...",
        "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII"
    ],
)
print(response)

import litellm
from litellm import embedding

litellm.vertex_project = "your-project-id"
litellm.vertex_location = "us-central1"

# Text + Image (GCS URL)
response = embedding(
    model="vertex_ai/gemini-embedding-2",
    input=[
        "Describe this image",
        "gs://my-bucket/images/photo.png"
    ],
)
print(response)

model_list:
  - model_name: gemini-embedding-2
    litellm_params:
      model: gemini/gemini-embedding-2
      api_key: os.environ/GEMINI_API_KEY
  - model_name: vertex-gemini-embedding-2
    litellm_params:
      model: vertex_ai/gemini-embedding-2
      vertex_project: os.environ/VERTEXAI_PROJECT
      vertex_location: global

general_settings:
  master_key: sk-1234

litellm --config config.yaml

curl -sS -X POST http://localhost:4000/v1/embeddings \
  -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-embedding-2",
    "input": [
      "The food was delicious and the waiter...",
      "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII"
    ]
  }'

Input Format Examples​

Supported MIME Types for Data URIs​

• Images: image/png, image/jpeg
• Audio: audio/mpeg, audio/wav
• Video: video/mp4, video/quicktime
• Documents: application/pdf

GCS URL MIME Inference​

For Vertex AI, MIME types are inferred from file extensions:

• .png → image/png
• .jpg / .jpeg → image/jpeg
• .mp3 → audio/mpeg
• .wav → audio/wav
• .mp4 → video/mp4
• .mov → video/quicktime
• .pdf → application/pdf

Optional Parameters​

response = embedding(
    model="gemini/gemini-embedding-2",
    input=["text to embed"],
    dimensions=768,  # Optional: control output vector size
)

GPT-5.5 is OpenAI's "smartest and most intuitive to use model" yet, with significant gains on agentic coding, computer use, and deep research workflows. Per OpenAI, it is a faster, sharper thinker for fewer tokens compared to GPT-5.4. GPT-5.5 Pro targets the most demanding reasoning tasks.

Usage​

model_list:
  - model_name: gpt-5.5
    litellm_params:
      model: openai/gpt-5.5
      api_key: os.environ/OPENAI_API_KEY
  - model_name: gpt-5.5-pro
    litellm_params:
      model: openai/gpt-5.5-pro
      api_key: os.environ/OPENAI_API_KEY

docker run -d \
  -p 4000:4000 \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  -v $(pwd)/config.yaml:/app/config.yaml \
  ghcr.io/berriai/litellm:v1.83.7-stable \
  --config /app/config.yaml

curl -X POST "http://0.0.0.0:4000/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_KEY" \
  -d '{
    "model": "gpt-5.5",
    "messages": [
      {"role": "user", "content": "Write a Python function to check if a number is prime."}
    ]
  }'

from litellm import completion

response = completion(
    model="openai/gpt-5.5",
    messages=[
        {"role": "user", "content": "Write a Python function to check if a number is prime."}
    ],
)

print(response.choices[0].message.content)

# GPT-5.5 Pro
response = completion(
    model="openai/gpt-5.5-pro",
    messages=[
        {"role": "user", "content": "Prove that the sum of two odd integers is even."}
    ],
)

print(response.choices[0].message.content)

Responses API​

For agentic and multi-turn workflows, use /v1/responses to preserve reasoning state and output item metadata across turns.

curl -X POST "http://0.0.0.0:4000/v1/responses" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_KEY" \
  -d '{
    "model": "gpt-5.5",
    "input": "Plan and write a Python script that scrapes a webpage and summarizes it."
  }'

Reasoning Effort​

reasoning_effort controls how much thinking the model applies. Supported values per model (verified against OpenAI's live API on 2026-04-24):

from litellm import completion

response = completion(
    model="openai/gpt-5.5",
    messages=[{"role": "user", "content": "Solve: what is the optimal strategy for..."}],
    reasoning_effort="high",
)

LiteLLM enforces these caps locally — passing an unsupported value (e.g. minimal) raises an UnsupportedParamsError instead of round-tripping to OpenAI for a 400.

Notes​

• For cost tracking on gpt-5.5 and gpt-5.5-pro, hit the Reload Model Cost Map button in the Admin UI (or POST /reload/model_cost_map). Works on any LiteLLM version v1.76.0 or newer — no container restart or image upgrade required.
• gpt-5.5-pro is a Responses API-only model (mode: "responses"). LiteLLM's Responses API bridge transparently translates completion() calls to /v1/responses, so the SDK example above works without code changes.
• GPT-5.5 supports reasoning, function calling, parallel tool calls, vision (image input), PDF input, prompt caching, web search, and structured output — see the OpenAI provider docs for advanced usage.
• Context window: 1.05M input tokens / 128K output tokens. Long-context tier pricing kicks in above 272K tokens.
• Azure availability: not yet — this post covers OpenAI direct only.

The fix landed in commit 7b7f304 (PR #25343) and has been in every release from v1.83.6-nightly onward. v1.83.7-stable includes it.

TLDR;​

• This was not exploitable by unauthenticated users. The affected endpoints (MCP server creation and the /mcp-rest/test/* preview endpoints) all sit behind LiteLLM's auth. An attacker needed a valid LiteLLM API key — and, with the patch, the PROXY_ADMIN role — before they could reach this code path.
• The fix has been live since v1.83.6-nightly. The first stable release with the fix is v1.83.7-stable. Full list of patched versions below.
• If you find other vulnerabilities, please send them our way. We run a bug bounty program and pay out for P0 (supply chain) and P1 (unauthenticated proxy access) issues. See our previous security update for the current bounty table.

What was the issue​

Per OX Security's advisory:

LiteLLM contains an authenticated remote command execution vulnerability in its MCP server creation functionality. The application allows users to add MCP servers via a JSON configuration specifying arbitrary command and args values. LiteLLM executes these values on the host without validation, enabling attackers to run arbitrary operating system commands.

Concretely: when adding an MCP server with transport: stdio, the command field was passed straight through to StdioServerParameters and executed as a subprocess on the proxy host. An authenticated user with permission to create MCP servers could run arbitrary commands as the LiteLLM process.

• CVE: CVE-2026-30623
• Severity: Critical
• Auth required: Yes (authenticated RCE, not unauthenticated)
• Affected surfaces:
  • MCP server creation/update (NewMCPServerRequest, UpdateMCPServerRequest)
  • /mcp-rest/test/connection and /mcp-rest/test/tools/list preview endpoints
  • Servers rehydrated from config or the DB at runtime

What the fix does​

Commit 7b7f304 lands four changes:

1. Command allowlist for stdio transport. A new constant MCP_STDIO_ALLOWED_COMMANDS restricts stdio command values to a small set of known MCP launchers:

   MCP_STDIO_ALLOWED_COMMANDS = frozenset(
       {"npx", "uvx", "python", "python3", "node", "docker", "deno"}
   )
   

   The list is extensible at deploy time via the LITELLM_MCP_STDIO_EXTRA_COMMANDS env var (comma-separated) if you need to allow additional binaries.

2. Pydantic-level validation. Both NewMCPServerRequest and UpdateMCPServerRequest now reject configs whose command basename is not in the allowlist — so the bad input never makes it past request parsing.

3. Defense-in-depth at runtime. _create_mcp_client re-validates the command when instantiating the stdio client, so any MCPServer reconstructed from an older DB row or config file (predating the allowlist) is also blocked at spawn time.

4. Locked down the preview endpoints. /mcp-rest/test/connection and /mcp-rest/test/tools/list now require the PROXY_ADMIN role. These "try before you add" endpoints were the easiest way to trigger command execution without persisting anything.

Versions with the fix​

The patch is present in every LiteLLM release tagged from v1.83.6-nightly onward. Confirmed tags at the time of publishing:

Any LiteLLM release newer than these also includes the fix.

What you should do​

• Upgrade. Move to v1.83.7-stable or later. If you track nightlies, anything >= v1.83.6-nightly is patched.
• Audit existing MCP servers. If you have stdio MCP servers configured from before the upgrade, any row whose command basename isn't in the allowlist will now fail to start. Either update the config to use an allowed launcher (e.g. npx, uvx, python) or add the binary to LITELLM_MCP_STDIO_EXTRA_COMMANDS.
• Review who has PROXY_ADMIN. The stdio test endpoints are now admin-only. If you'd previously delegated MCP testing to non-admin users, they'll now hit a 403.

Credit​

Thanks to the OX Security research team — Moshe Siman Tov Bustan, Mustafa Naamnih, and Nir Zadok — for the disclosure. Their full cross-ecosystem writeup is here.

If you find a security issue in LiteLLM, please report it through our bug bounty program.

Akto now runs natively inside the LiteLLM proxy as a chained guardrail.

LiteLLM already ships with built-in guardrails for fast, deterministic checks (regex-based PII, secret scanning, banned-word lists). Akto adds a second layer on top — model-based detection for the cases deterministic rules can't cover: prompt injection, semantic PII leaks, and custom policy violations that require an LLM to classify intent.

You run them together. LiteLLM's guardrails handle the cheap, fast checks; Akto handles the scenarios that need a model in the loop.

Akto runs in sync mode (block on violation before the LLM is called) or async mode (log and alert without adding latency). Configure it as a callback on your existing proxy — no app-level changes.

Get started: Akto guardrail setup guide

Read the full announcement on Akto's blog →

Docker Image​

docker pull ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.83.3-stable.opus-4.7

Usage - Anthropic​

model_list:
  - model_name: claude-opus-4-7
    litellm_params:
      model: anthropic/claude-opus-4-7
      api_key: os.environ/ANTHROPIC_API_KEY

docker run -d \
  -p 4000:4000 \
  -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
  -v $(pwd)/config.yaml:/app/config.yaml \
  ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.83.3-stable.opus-4.7 \
  --config /app/config.yaml

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
  "model": "claude-opus-4-7",
  "messages": [
    {
      "role": "user",
      "content": "what llm are you"
    }
  ]
}'

Usage - Azure​

model_list:
  - model_name: claude-opus-4-7
    litellm_params:
      model: azure_ai/claude-opus-4-7
      api_key: os.environ/AZURE_AI_API_KEY
      api_base: os.environ/AZURE_AI_API_BASE  # https://<resource>.services.ai.azure.com

docker run -d \
  -p 4000:4000 \
  -e AZURE_AI_API_KEY=$AZURE_AI_API_KEY \
  -e AZURE_AI_API_BASE=$AZURE_AI_API_BASE \
  -v $(pwd)/config.yaml:/app/config.yaml \
  ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.83.3-stable.opus-4.7 \
  --config /app/config.yaml

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
  "model": "claude-opus-4-7",
  "messages": [
    {
      "role": "user",
      "content": "what llm are you"
    }
  ]
}'

Usage - Vertex AI​

model_list:
  - model_name: claude-opus-4-7
    litellm_params:
      model: vertex_ai/claude-opus-4-7
      vertex_project: os.environ/VERTEX_PROJECT
      vertex_location: us-east5

docker run -d \
  -p 4000:4000 \
  -e VERTEX_PROJECT=$VERTEX_PROJECT \
  -e GOOGLE_APPLICATION_CREDENTIALS=/app/credentials.json \
  -v $(pwd)/config.yaml:/app/config.yaml \
  -v $(pwd)/credentials.json:/app/credentials.json \
  ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.83.3-stable.opus-4.7 \
  --config /app/config.yaml

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
  "model": "claude-opus-4-7",
  "messages": [
    {
      "role": "user",
      "content": "what llm are you"
    }
  ]
}'

Usage - Bedrock​

model_list:
  - model_name: claude-opus-4-7
    litellm_params:
      model: bedrock/anthropic.claude-opus-4-7
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: us-east-1

docker run -d \
  -p 4000:4000 \
  -e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
  -e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
  -v $(pwd)/config.yaml:/app/config.yaml \
  ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.83.3-stable.opus-4.7 \
  --config /app/config.yaml

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
  "model": "claude-opus-4-7",
  "messages": [
    {
      "role": "user",
      "content": "what llm are you"
    }
  ]
}'

Advanced Features​

Adaptive Thinking​

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
  "model": "claude-opus-4-7",
  "messages": [
    {
      "role": "user",
      "content": "Solve this complex problem: What is the optimal strategy for..."
    }
  ],
  "reasoning_effort": "high"
}'

curl --location 'http://0.0.0.0:4000/v1/messages' \
--header 'x-api-key: sk-12345' \
--header 'content-type: application/json' \
--data '{
    "model": "claude-opus-4-7",
    "max_tokens": 16000,
    "thinking": {
        "type": "adaptive"
    },
    "messages": [
        {
            "role": "user",
            "content": "Explain why the sum of two even numbers is always even."
        }
    ]
}'

Effort Levels​

Claude Opus 4.7 supports four effort levels: low, medium, high (default), and xhigh. These give you finer-grained control over how much reasoning the model applies to a task. Pass the effort level via the output_config parameter.

xhigh is a new effort level introduced with Opus 4.7 that sits above high. The max effort level is Claude Opus 4.6 only and is not available on 4.7.

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
  "model": "claude-opus-4-7",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing"
    }
  ],
  "output_config": {
    "effort": "xhigh"
  }
}'

import openai

client = openai.OpenAI(
    api_key="your-litellm-key",
    base_url="http://0.0.0.0:4000"
)

response = client.chat.completions.create(
    model="claude-opus-4-7",
    messages=[{"role": "user", "content": "Explain quantum computing"}],
    extra_body={"output_config": {"effort": "xhigh"}}
)

from litellm import completion

response = completion(
    model="anthropic/claude-opus-4-7",
    messages=[{"role": "user", "content": "Explain quantum computing"}],
    output_config={"effort": "xhigh"},
)

curl --location 'http://0.0.0.0:4000/v1/messages' \
--header 'x-api-key: sk-12345' \
--header 'content-type: application/json' \
--data '{
    "model": "claude-opus-4-7",
    "max_tokens": 4096,
    "messages": [
        {
            "role": "user",
            "content": "Explain quantum computing"
        }
    ],
    "output_config": {
        "effort": "xhigh"
    }
}'

Effort level guide:

Enterprise AI Gateway deployments put Redis in the hot path for nearly every request: rate limiting, cache lookups, spend tracking. When Redis is healthy, the latency contribution is single-digit milliseconds — invisible to end users. When it degrades, a production AI Gateway needs to stay up regardless.

Running LiteLLM at scale across 100+ pods means designing for failure modes before they appear. The easy case is Redis going fully down: fail fast, fall through to the database, continue serving requests. The hard case — the one that takes down gateways — is a slow Redis: still accepting connections, still responding, but timing out after 20-30 seconds per operation.

Why slow Redis is harder than a full outage​

With 100 pods each hanging 30 seconds on every auth check, threadpools fill up and requests queue. By the time Redis times out and falls through to Postgres, the database receives 100× its normal load from simultaneous fallbacks. A slow Redis becomes a database outage becomes a full gateway outage. A production-grade AI Gateway cannot allow one degraded dependency to cascade into total failure.

The fix: circuit breaker pattern​

The circuit breaker pattern tracks consecutive failures and cuts off the unhealthy dependency before it cascades. Instead of hanging 30 seconds on each Redis call, the circuit opens after 5 consecutive failures and fast-fails at 0ms — no network call, no wait.

Three states:

• CLOSED — normal. All Redis calls pass through.
• OPEN — Redis is unhealthy. Every call fast-fails instantly. Requests continue with degraded-but-functional behavior: auth and rate limiting fall back to the database.
• HALF-OPEN — after 60 seconds, one probe request tests recovery. Success closes the circuit; failure resets the timer.

This is how a reliable AI Gateway handles infrastructure degradation: stay up, degrade gracefully, recover automatically.

How requests flow through the AI Gateway​

When the circuit is open, the gateway does not stall. Auth checks fall back to Postgres — slower, but bounded. The database absorbs the load because it receives some requests via DB fallback, not all 100 pods simultaneously dumping their queued requests after a 30-second timeout.

The difference between a resilient AI Gateway and a fragile one: controlled degradation vs. uncontrolled cascade.

The implementation​

class RedisCircuitBreaker:
    def __init__(self, failure_threshold: int, recovery_timeout: int):
        self.failure_threshold = failure_threshold  # default: 5
        self.recovery_timeout = recovery_timeout    # default: 60s
        self._failure_count = 0
        self._state = self.CLOSED

    def is_open(self) -> bool:
        if self._state == self.OPEN:
            if time.time() - self._opened_at > self.recovery_timeout:
                self._state = self.HALF_OPEN
                return False  # this caller is the recovery probe
            return True       # fast-fail
        return False

    def record_failure(self):
        self._failure_count += 1
        self._opened_at = time.time()
        if self._failure_count >= self.failure_threshold:
            self._state = self.OPEN  # open the circuit

    def record_success(self):
        self._failure_count = 0
        self._state = self.CLOSED   # Redis recovered

Every async Redis operation goes through a decorator that checks the breaker before touching the network. When open, it raises immediately:

@_redis_circuit_breaker_guard
async def async_get_cache(self, key: str):
    ...

The decorator handles all bookkeeping — success resets nothing, failures increment the counter, exceptions trigger record_failure(). The caller sees a clean exception and falls through to its normal non-Redis path. No changes required in calling code.

AI Gateway resilience in production​

Redis degradation events no longer cascade in production. The observable symptom during a Redis slowdown is a temporary bump in cache miss rate — the right failure mode for a resilient AI Gateway. Auth still works. Rate limiting still works. Spend tracking still works, at slightly higher DB cost. Recovery is fully automatic when Redis comes back.

# configure via environment variables
REDIS_CIRCUIT_BREAKER_FAILURE_THRESHOLD=5   # failures before opening
REDIS_CIRCUIT_BREAKER_RECOVERY_TIMEOUT=60  # seconds before probe

The circuit breaker ships on by default in all LiteLLM versions since v1.82.0. No configuration needed for most deployments.

Key Takeaways​

• A slow Redis is more dangerous than a downed one: 30-second timeouts across 100+ pods overwhelm Postgres at 100× normal load
• LiteLLM's AI Gateway uses a circuit breaker that fast-fails Redis calls at 0ms after 5 consecutive failures
• Three states: CLOSED (normal), OPEN (fast-fail + DB fallback), HALF-OPEN (probe recovery)
• Auth, rate limiting, and spend tracking continue working during Redis outages
• Resilient, production-grade behavior — enabled by default since v1.82.0, no configuration required

Frequently Asked Questions​

Does the circuit breaker affect normal Redis performance?​

No. When Redis is healthy (circuit CLOSED), every call passes through with zero overhead. The breaker only activates after 5 consecutive failures — transparent under normal conditions.

What happens to rate limiting when the circuit is open?​

Rate limiting falls back to Postgres with bounded load. Limits remain enforced at slightly higher DB cost until Redis recovers and the circuit closes automatically.

How is this different from basic Redis retry logic?​

Retry logic still waits for each timeout (30s × retries). The circuit breaker cuts the connection immediately at 0ms after the failure threshold, preventing threadpool exhaustion across all pods simultaneously. Retries make slow-Redis worse; the circuit breaker contains it.

Is this available in LiteLLM OSS?​

Yes. The circuit breaker ships in LiteLLM OSS (Apache 2.0) by default since v1.82.0. LiteLLM Enterprise adds SSO/SCIM, air-gapped deployment, 24/7 SLA support, and advanced guardrails on top of the OSS foundation.

Conclusion​

Redis resilience is one layer of what makes LiteLLM a production-grade, reliable AI Gateway at scale. The circuit breaker pattern ensures infrastructure degradation stays contained — the right failure mode is a temporary cache miss rate bump, not a full outage. This is how AI Gateway infrastructure should behave under pressure: degrade gracefully, recover automatically, keep serving traffic. For teams with strict uptime and compliance requirements, LiteLLM Enterprise provides the additional controls needed for regulated production environments.

Recommended Reading​

• LiteLLM AI Gateway — full feature overview
• Load balancing and routing across 100+ LLM providers
• Spend tracking and budget controls

We used the session to share our CI/CD v2 improvements, product stability work, and what we are prioritizing next across reliability and product roadmap.

CI/CD v2 improvements​

Our CI/CD v2 work is centered around four goals:

1. Limit what each package can access
2. Reduce the number of sensitive environment variables
3. Avoid compromised packages
4. Reduce the risk of release tampering

New architecture: isolated environments​

We have begun moving to isolated environments for distinct CI/CD stages to reduce the chance that a single compromised step can inherit broad access across the entire pipeline.

Current rollout status​

These changes are deployed in our current release workflow. See here

Independently verify releases​

A key part of CI/CD v2 is supporting independent verification of release artifacts using our published verification process, while reducing reliance on any single credential or release path.

Learn more about how to verify releases

Stability improvements​

SDLC improvements​

This month, we're focusing on process stability improvements around:

• Improving main-branch stability
• Mapping UI QA to built Docker images for 1:1 environment parity
• Consistent release tags across PyPI and Docker
• Fixing release notes publication

Improving main-branch stability​

We're introducing a staging-gated flow:

• Only an internal staging branch can push to main.
• PRs to that staging branch must pass CircleCI LLM API testing.
• Collision handling happens on staging, which is designed to reduce unstable changes reaching main.

UI QA in Docker environment​

Moving forward, all UI QA will be performed in the built Docker image that users run.

Previously, some UI QA paths were run in local environments that did not fully replicate Docker runtime conditions.

That contributed to release-specific issues, including MCP registration problems in v1.82.3.

Consistent release tags​

Today we publish releases for multiple scenarios:

• Dev (Built of a PR for a customer-specific scenario)
• Nightly (Passes all CI/CD checks)
• Release Candidate (Passes all CI/CD checks + manual UI QA)
• Stable (intended to pass all CI/CD checks + manual UI QA + 7 days of production testing)

We are targeting a consistent naming convention across PyPI and Docker by the end of April.

Release notes​

CI/CD v2 changes moved release notes to a manual path. This is a temporary solution while we investigate a better automated workflow. We are targeting a more consistent process by the end of April.

Product stability improvements​

Stable Prisma migrations​

Today, we have observed several migration failure classes:

• Migration not applied
• Migration marked applied but incomplete
• Migration not applied due to non-root image issues

We're prioritizing this work this month and have assigned an engineering owner to the effort. Our target is to resolve these error classes by the end of April.

UI type safety​

Another area of focus is improving the stability of the UI. Today, one cause of errors is that the UI maintains its own assumptions about backend API types. This can lead to issues when backend responses differ from UI assumptions.

We aim to move to having the UI and Backend be in sync with each other, and are exploring OpenAPI-driven mapping to achieve this.

Product roadmap​

Our Assumptions​

Over the next few years, we expect:

• Companies will give employees more AI tools.
• More AI agents will move into production workflows across HR, finance, support, and operations.

Our Inferences​

Near-term​

• AI spend will increase.
• Uptime and latency will become even more important.
• More AI resources (skills, CLIs, and related assets) will require governance.
• Agent and MCP usage patterns will require deeper controls.
• Broader developer adoption will increase the need for simpler, more discoverable tooling.

Long-term​

• We expect many organizations to treat agent auditability (how decisions were made across LLM + MCP + sub-agent inputs/outputs) as a compliance expectation.
• Permission management will get more complex as user-agent interaction chains deepen.

Roadmap timelines in this post are targets and may evolve based on validation and user feedback.

April investments​

Reliability​

• Increase uptime for 10k+ RPS scenarios.
• Investigate latency overhead for long-running Claude Code requests.

Feature reliability​

• Polish MCP authentication.
• Better understand how teams are using agents through LiteLLM.

Governance​

• Launch Skills as a first-class citizen in LiteLLM.

Q&A​

Thank you again for all the questions and direct feedback. We will keep sharing concrete progress updates as these efforts ship.

Hiring​

We are actively hiring across several roles, please apply here if you're interested!

We've also launched a bug bounty program and Veria Labs is continuing to audit the proxy. More fixes will ship in upcoming versions.

The two high-severity issues (CVE-2026-35029 and GHSA-69x8-hrgq-fjj8) both require the attacker to already have a valid API key for the proxy. These are not exploitable by unauthenticated users.

The critical-severity issue (CVE-2026-35030) is an authentication bypass, but only affects deployments with enable_jwt_auth explicitly enabled, which is off by default. The default LiteLLM configuration is not affected, and no LiteLLM Cloud customers had this feature enabled.

Vulnerabilities​

CVE-2026-35030: Authentication bypass via OIDC cache collision (Critical)​

Found by Veria Labs.

When enable_jwt_auth is enabled, LiteLLM cached OIDC userinfo using token[:20] as the cache key. JWTs from the same signing algorithm share the same header prefix, so an attacker could forge a token that hits another user's cache entry and inherit their session. We fixed this by keying the cache on sha256(token) instead.

Most deployments are not affected. This requires enable_jwt_auth: true, which is off by default. If you can't upgrade, disable JWT auth as a workaround.

Full advisory: GHSA-jjhc-v7c2-5hh6

CVE-2026-35029: Privilege escalation via /config/update (High)​

Found by Lakera.

/config/update didn't check the caller's role. Any authenticated user could modify the proxy's runtime configuration, which could lead to arbitrary file read, admin account takeover, or remote code execution. We now require the proxy_admin role on this endpoint.

Full advisory: GHSA-53mr-6c8q-9789

Password hash exposure and pass-the-hash login (High)​

Weak hashing originally reported by GitHub user hamzayevmaqsud (#15484). The full chain was identified by Luca Vandenweghe and Maarten De Rammelaere of iO Digital.

Passwords were stored as unsalted SHA-256 hashes, and in some cases plaintext. Several API endpoints returned the hash to any authenticated user, and /v2/login accepted the raw hash as a credential without re-hashing it, so a stolen hash was as good as the password itself. We've moved to scrypt with random salts and stripped hashes from all API responses.

Full advisory: GHSA-69x8-hrgq-fjj8

Bug bounty program​

After the supply chain incident and these disclosures it was clear we needed more external eyes on the project. We've set up a bug bounty program so researchers have a way to report issues.

Bounties are currently paid for P0 (supply chain) and P1 (unauthenticated proxy access) vulnerabilities:

We plan on expanding the program further in the coming months. More info about the bug bounty program is available here.

What's next​

Veria Labs is continuing to work with us on a broader audit of the proxy. Security advisories sent through Github will be responded to within five business days. We'll publish advisories as issues are confirmed and fixed.

Agenda​

• Product updates and roadmap progress
• Reliability and security updates
• Open Q&A with the team

How to contribute​

Add your thoughts to this ticket to help us shape the agenda.

Register​

Register here: LiteLLM April Townhall Form

We will hold the townhall from 7:30 AM to 8:30 AM PST on Zoom.

For security, attendance is restricted to corporate emails. If you register with a non-corporate email, we will share the townhall slides and accompanying blog post after the event.

Building on the roadmap from our security incident, CI/CD v2 introduces isolated environments, stronger security gates, and safer release separation for LiteLLM.

What changed​

• Security scans and unit tests run in isolated environments.
• Validation and release are separated into different repositories, making it harder for an attacker to reach release credentials.
• Trusted Publishing for PyPI releases - this means no long-lived credentials are used to publish releases.
• Immutable Docker release tags - this means no tampering of Docker release tags after they are published Learn more. Note: work for GHCR docker releases is planned as well.
• Docker image signing with Cosign - all release images are signed so users can independently verify they came from us.

Verify Docker image signatures​

Starting from v1.83.0-nightly, all LiteLLM Docker images published to GHCR are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:<release-tag>

Verify using a release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/<release-tag>/cosign.pub \
  ghcr.io/berriai/litellm:<release-tag>

Replace <release-tag> with the version you are deploying (e.g. v1.83.0-stable).

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's next​

Moving forward, we plan on:

• Adopting OpenSSF (this is a set of security criteria that projects should meet to demonstrate a strong security posture - Learn more)

  • We've added Scorecard and Allstar to our Github

• Adding SLSA Build Provenance to our CI/CD pipeline - this means we allow users to independently verify that a release came from us and prevent silent modifications of releases after they are published.

We hope that this will mean you can be confident that the releases you are using are safe and from us.

The principle​

The new CI/CD pipeline reflects the principles, outlined below, and is designed to be more secure and reliable:

• Limit what each package can access
• Reduce the number of sensitive environment variables
• Avoid compromised packages
• Prevent release tampering

How to help:​

Help us plan April's stability sprint - https://github.com/BerriAI/litellm/issues/24825

We are partnering with Vanta to recertify LiteLLM's compliance for SOC 2 Type 2 and ISO 27001.

As part of this process, we are also identifying independent auditors to validate and verify our compliance posture.

This is part of our commitment to being the most secure and transparent AI Gateway possible.

We wanted to use that time to walk through what we know, what we've done so far, and how we're improving LiteLLM's release and security processes going forward. This post is a written version of that update. Slides available here

What happened​

On March 24, 2026 at 10:39 UTC, LiteLLM v1.82.7 was pushed to PyPI. Version v1.82.8 was published soon after. Those packages were live for about 40 minutes before being quarantined by PyPI. By 16:00 UTC, the LiteLLM team had worked with PyPI to delete the affected packages.

At this point, our understanding is that this was a supply-chain incident affecting those two published versions.

How did this happen?​

Our understanding is that the issue came from the compromised Trivy security scanner dependency in our CI/CD pipeline.

There were three major contributing factors:

1. Shared CI/CD environment​

At the time, everything was running on CircleCI, and all steps shared a common environment. That increased blast radius: if one component was compromised, it could potentially access credentials or context intended for other parts of the pipeline.

2. Static credentials in environment variables​

Release credentials, including credentials for PyPI, GHCR, and Docker publishing, were available as static secrets in the environment. That meant a compromised step could access long-lived release credentials.

3. Unpinned Trivy dependency​

In our security scanning component, we had an unpinned Trivy dependency. Our present understanding is that a compromised Trivy package ran during the scan, had access to environment variables, and enabled attackers to obtain those credentials.

In summary: a compromised package in CI had access to secrets it should not have had, and those secrets were then used in the release path.

What we've already done​

In the last 3 days, we've taken the following steps:

1. Minimize Scope of Impact​

Prevented further key abuse​

We deleted or rotated all impacted or adjacent secret keys, including PyPI, GitHub, Docker, and related credentials. Out of an abundance of caution, we've also rotated LiteLLM maintainer accounts.

Prevent branch attacks​

We removed roughly 6,000 open branches and added an auto-deletion policy for branches merged into main. This reduces the surface area for branch-based abuse.

Pinned CI/CD dependencies​

We've pinned all Github Actions, and are working on pinning all CircleCI dependencies as well.

Paused releases​

We've paused new releases until we've confirmed codebase security and put stronger release controls in place.

2. Secured LiteLLM​

Forensic analysis​

We are working with Google's Mandiant cybersecurity team to confirm the source of the attack and verify the security of the codebase. We also confirmed that no malicious code was pushed to main.

Confirm Application Security​

In parallel, we are working with whitehat hackers at Veria Labs to verify application security and review improvements to our CI/CD process.

We have also confirmed that the last 20 LiteLLM releases contain no indicators of compromise, and that no unauthenticated attacks can be made against LiteLLM Proxy based on our current investigation. Check Security Blog for release verification.

Created a security working group​

We created a new security working group inside LiteLLM focused on:

• Building threat models
• Auditing the build process and dependencies

If you're interested in joining the security working group, please file an issue here.

3. Improved CI/CD​

We've already begun making structural changes to how releases are built and published. These align with our goals (covered in the next section) around isolated environments, ephemeral credentials, and release auditing.

Roadmap​

We plan on following 4 guiding principles for our new CI/CD pipeline:

1. Limit what each package can access
2. Reduce the number of sensitive environment variables
3. Avoid compromised packages
4. Prevent release tampering

Isolated environments​

We are breaking our CI/CD into 4 semantic concepts:

1. Unit tests
2. Integration tests
3. Security scans
4. Release publishing

And will be running each of these in isolated environments.

This will limit the damage that any single compromised component can cause.

Ephemeral credentials​

We plan to move to ephemeral credentials for PyPI (Trusted Publisher) and GHCR (Token-based authentication) releases. This will reduce the risk of credentials being leaked or compromised.

We have already begun doing this:

• PyPI Trusted Publisher on GitHub Actions PR
• GHCR Token-based authentication on GitHub Actions PR

Release auditing​

Our goal is to allow users to independently verify that a release came from us and prevent silent modifications of releases after they are published.

This will ensure, your releases are safe, even when:

• Stolen PyPI/GHCR credentials are used to publish malicious releases
• Tampered registry artifacts are published
• Tag mutations are made after the release is published

We believe that Cosign is a good fit for this, and have shipped it in PR #24683.

How to verify a Docker image with Cosign​

Starting from v1.83.0-nightly, all LiteLLM Docker images published to GHCR are signed with cosign. Every release is signed with the same key that was introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:<release-tag>

Verify using a release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/<release-tag>/cosign.pub \
  ghcr.io/berriai/litellm:<release-tag>

Replace <release-tag> with the version you are deploying (e.g. v1.83.0-stable).

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

Avoid Compromised Packages​

• Move to pinned, verified SHAs for packages and actions used in CI/CD, avoiding latest wherever possible.
• Add a cooldown period before upgrading to a new version of a package - allows more time to investigate and verify the new version.

We've added zizmor to help us catch issues such as unpinned dependencies and credential leakage. commit.

Frequently Asked Questions​

Q: Did you observe any lateral movement into your corporate environment during this incident?

A: No. Our investigation to date, conducted in coordination with external security experts, has found no evidence of lateral movement into our internal corporate systems. The incident was isolated to the CI/CD pipeline and the release path for specific versions (v1.82.7 and v1.82.8). As a proactive measure, we have rotated all potentially impacted or adjacent secrets—including PyPI, GitHub, and Docker credentials—and updated maintainer account security to ensure continued isolation.

Q: Do you expect delays in future product releases due to these new security measures?

A: We are committed to balancing security with speed. While we have temporarily paused releases to implement stronger controls, we are moving quickly to automate our new security protocols. We are currently implementing isolated CI/CD environments, ephemeral credentials (via Trusted Publishers), and release auditing with Cosign. These improvements are designed to be integrated into our automated pipeline, allowing us to maintain a fast release cadence while ensuring every package is verified and secure.

Q: Were older packages impacted?

Our current findings show no indicators of compromise in the last 20 versions of LiteLLM. This was manually verified by our team and independently reviewed by Veria Labs.

We have also published the verified versions for users to use. Check Security Blog for release verification.

Questions & Support​

If you believe your systems may be affected, contact us immediately:

• Security: security@berri.ai
• Support: support@berri.ai
• Slack: Reach out to the LiteLLM team directly here

Hiring​

We are currently hiring for:

• DevOps Engineer - to keep ci/cd secure and running smoothly
• Security Engineer - to keep the application secure

If you're interest in joining, please apply here

Status: Active investigation Last updated: March 27, 2026

Update (March 30): A new clean version of LiteLLM is now available (v1.83.0). This was released by our new CI/CD v2 pipeline which added isolated environments, stronger security gates, and safer release separation for LiteLLM.

Update (March 27): Review Townhall updates, including explanation of the incident, what we've done, and what comes next. Learn more

Update (March 27): Added Verified safe versions section with SHA-256 checksums for all audited PyPI and Docker releases.

Update (March 26): Added checkmarx[.]zone to Indicators of compromise

Update (March 25): Added community-contributed scripts for scanning GitHub Actions and GitLab CI pipelines for the compromised versions. See How to check if you are affected. s/o @Zach Fury for these scripts.

TLDR;​

• The compromised PyPI packages were litellm==1.82.7 and litellm==1.82.8. Those packages were live on March 24, 2026 from 10:39 UTC for about 40 minutes before being quarantined by PyPI.
• We believe that the compromise originated from the Trivy dependency used in our CI/CD security scanning workflow.
• Customers running the official LiteLLM Proxy Docker image were not impacted. That deployment path pins dependencies in requirements.txt and does not rely on the compromised PyPI packages.
• We have paused all new LiteLLM releases until we complete a broader supply-chain review and confirm the release path is safe. Updated: We have now released a new safe version of LiteLLM (v1.83.0) by our new CI/CD v2 pipeline which added isolated environments, stronger security gates, and safer release separation for LiteLLM. We have also verified the codebase is safe and no malicious code was pushed to main.

Overview​

LiteLLM AI Gateway is investigating a suspected supply chain attack involving unauthorized PyPI package publishes. Current evidence suggests a maintainer's PyPI account may have been compromised and used to distribute malicious code.

At this time, we believe this incident may be linked to the broader Trivy security compromise, in which stolen credentials were reportedly used to gain unauthorized access to the LiteLLM publishing pipeline.

This investigation is ongoing. Details below may change as we confirm additional findings.

Confirmed affected versions​

The following LiteLLM versions published to PyPI were impacted:

• v1.82.7: contained a malicious payload in the LiteLLM AI Gateway proxy_server.py
• v1.82.8: contained litellm_init.pth and a malicious payload in the LiteLLM AI Gateway proxy_server.py

If you installed or ran either of these versions, review the recommendations below immediately.

Note: These versions have already been removed from PyPI.

What happened​

Initial evidence suggests the attacker bypassed official CI/CD workflows and uploaded malicious packages directly to PyPI.

These compromised versions appear to have included a credential stealer designed to:

• Harvest secrets by scanning for:
  • environment variables
  • SSH keys
  • cloud provider credentials (AWS, GCP, Azure)
  • Kubernetes tokens
  • database passwords
• Encrypt and exfiltrate data via a POST request to models.litellm.cloud, which is not an official BerriAI / LiteLLM domain

Who is affected​

You may be affected if any of the following are true:

• You installed or upgraded LiteLLM via pip on March 24, 2026, between 10:39 UTC and 16:00 UTC
• You ran pip install litellm without pinning a version and received v1.82.7 or v1.82.8
• You built a Docker image during this window that included pip install litellm without a pinned version
• A dependency in your project pulled in LiteLLM as a transitive, unpinned dependency (for example through AI agent frameworks, MCP servers, or LLM orchestration tools)

You are not affected if any of the following are true:

LiteLLM AI Gateway/Proxy users: Customers running the official LiteLLM Proxy Docker image were not impacted. That deployment path pins dependencies in requirements.txt and does not rely on the compromised PyPI packages.

• You are using LiteLLM Cloud
• You are using the official LiteLLM AI Gateway Docker image: ghcr.io/berriai/litellm
• You are on v1.82.6 or earlier and did not upgrade during the affected window
• You installed LiteLLM from source via the GitHub repository, which was not compromised

How to check if you are affected​

pip show litellm

export GITHUB_TOKEN="your-github-pat"

python find_litellm_github.py

#!/usr/bin/env python3
"""
Scan all GitHub Actions jobs in a GitHub org that ran between
0800-1244 UTC today and identify any that installed litellm 1.82.7 or 1.82.8.

Adjust WINDOW_START / WINDOW_END to cover March 24, 2026 if running later.
"""

import io
import os
import re
import sys
import zipfile
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import datetime, timezone

import requests

GITHUB_URL   = "https://api.github.com"
ORG          = "your-org"  # <-- set to your GitHub organization
TOKEN        = os.environ.get("GITHUB_TOKEN", "")

TODAY        = datetime.now(timezone.utc).date()
WINDOW_START = datetime(TODAY.year, TODAY.month, TODAY.day,  8,  0, 0, tzinfo=timezone.utc)
WINDOW_END   = datetime(TODAY.year, TODAY.month, TODAY.day, 12, 44, 0, tzinfo=timezone.utc)

TARGET_VERSIONS = {"1.82.7", "1.82.8"}
VERSION_PATTERN = re.compile(r"litellm[=\-](\d+\.\d+\.\d+)", re.IGNORECASE)

SESSION = requests.Session()
SESSION.headers.update({
    "Authorization": f"Bearer {TOKEN}",
    "Accept": "application/vnd.github+json",
    "X-GitHub-Api-Version": "2022-11-28",
})


def get_paginated(url, params=None):
    params = dict(params or {})
    params.setdefault("per_page", 100)
    page = 1
    while True:
        params["page"] = page
        resp = SESSION.get(url, params=params, timeout=30)
        if resp.status_code == 404:
            return
        resp.raise_for_status()
        data = resp.json()
        if isinstance(data, dict):
            items = next((v for v in data.values() if isinstance(v, list)), [])
        else:
            items = data
        if not items:
            break
        yield from items
        if len(items) < params["per_page"]:
            break
        page += 1


def parse_ts(ts_str):
    if not ts_str:
        return None
    return datetime.fromisoformat(ts_str.replace("Z", "+00:00"))


def get_repos():
    repos = []
    for r in get_paginated(f"{GITHUB_URL}/orgs/{ORG}/repos", {"type": "all"}):
        repos.append({"id": r["id"], "name": r["name"], "full_name": r["full_name"]})
    return repos


def get_runs_in_window(repo_full_name):
    created_filter = (
        f"{WINDOW_START.strftime('%Y-%m-%dT%H:%M:%SZ')}"
        f"..{WINDOW_END.strftime('%Y-%m-%dT%H:%M:%SZ')}"
    )
    url = f"{GITHUB_URL}/repos/{repo_full_name}/actions/runs"
    runs = []
    for run in get_paginated(url, {"created": created_filter, "per_page": 100}):
        ts = parse_ts(run.get("run_started_at") or run.get("created_at"))
        if ts and WINDOW_START <= ts <= WINDOW_END:
            runs.append(run)
    return runs


def get_jobs_for_run(repo_full_name, run_id):
    url = f"{GITHUB_URL}/repos/{repo_full_name}/actions/runs/{run_id}/jobs"
    jobs = []
    for job in get_paginated(url, {"filter": "all"}):
        ts = parse_ts(job.get("started_at"))
        if ts and WINDOW_START <= ts <= WINDOW_END:
            jobs.append(job)
    return jobs


def fetch_job_log(repo_full_name, job_id):
    url = f"{GITHUB_URL}/repos/{repo_full_name}/actions/jobs/{job_id}/logs"
    resp = SESSION.get(url, timeout=60, allow_redirects=True)
    if resp.status_code in (403, 404, 410):
        return ""
    resp.raise_for_status()

    content_type = resp.headers.get("Content-Type", "")
    if "zip" in content_type or resp.content[:2] == b"PK":
        try:
            with zipfile.ZipFile(io.BytesIO(resp.content)) as zf:
                parts = []
                for name in sorted(zf.namelist()):
                    with zf.open(name) as f:
                        parts.append(f.read().decode("utf-8", errors="replace"))
                return "\n".join(parts)
        except zipfile.BadZipFile:
            pass
    return resp.text


def check_job(repo_full_name, job):
    job_id   = job["id"]
    job_name = job["name"]
    run_id   = job["run_id"]
    started  = job.get("started_at", "")

    log_text = fetch_job_log(repo_full_name, job_id)
    if not log_text:
        return None

    found_versions = set()
    context_lines  = []
    for line in log_text.splitlines():
        m = VERSION_PATTERN.search(line)
        if m:
            ver = m.group(1)
            if ver in TARGET_VERSIONS:
                found_versions.add(ver)
                context_lines.append(line.strip())

    if not found_versions:
        return None

    return {
        "repo":       repo_full_name,
        "run_id":     run_id,
        "job_id":     job_id,
        "job_name":   job_name,
        "started_at": started,
        "versions":   sorted(found_versions),
        "context":    context_lines[:10],
        "job_url":    job.get("html_url", f"https://github.com/{repo_full_name}/actions/runs/{run_id}"),
    }


def main():
    if not TOKEN:
        print("ERROR: Set GITHUB_TOKEN environment variable.", file=sys.stderr)
        sys.exit(1)

    print(f"Time window : {WINDOW_START.isoformat()} -> {WINDOW_END.isoformat()}")
    print(f"Hunting for : litellm {', '.join(sorted(TARGET_VERSIONS))}")
    print()

    print(f"Fetching repositories for org '{ORG}'...")
    repos = get_repos()
    print(f"  Found {len(repos)} repositories")
    print()

    jobs_to_check = []

    print("Scanning workflow runs for time window...")
    for repo in repos:
        full_name = repo["full_name"]
        try:
            runs = get_runs_in_window(full_name)
        except requests.HTTPError as e:
            print(f"  WARN: {full_name} - {e}", file=sys.stderr)
            continue
        if not runs:
            continue
        print(f"  {full_name}: {len(runs)} run(s) in window")
        for run in runs:
            try:
                jobs = get_jobs_for_run(full_name, run["id"])
            except requests.HTTPError as e:
                print(f"    WARN: run {run['id']} - {e}", file=sys.stderr)
                continue
            for job in jobs:
                jobs_to_check.append((full_name, job))

    total = len(jobs_to_check)
    print(f"\nFetching logs for {total} job(s)...")
    print()

    hits = []
    with ThreadPoolExecutor(max_workers=8) as pool:
        futures = {
            pool.submit(check_job, full_name, job): (full_name, job["id"])
            for full_name, job in jobs_to_check
        }
        done = 0
        for future in as_completed(futures):
            done += 1
            full_name, jid = futures[future]
            try:
                result = future.result()
            except Exception as e:
                print(f"  ERROR {full_name} job {jid}: {e}", file=sys.stderr)
                continue
            if result:
                hits.append(result)
            print(
                f"  [{done}/{total}] {full_name} job {jid}" +
                (f"  *** HIT: litellm {result['versions']} ***" if result else ""),
                flush=True,
            )

    print()
    print("=" * 72)
    print(f"RESULTS: {len(hits)} job(s) installed litellm {' or '.join(sorted(TARGET_VERSIONS))}")
    print("=" * 72)

    if not hits:
        print("No matches found.")
        return

    for h in sorted(hits, key=lambda x: x["started_at"]):
        print()
        print(f"  Repo      : {h['repo']}")
        print(f"  Job       : {h['job_name']} (#{h['job_id']})")
        print(f"  Run ID    : {h['run_id']}")
        print(f"  Started   : {h['started_at']}")
        print(f"  Versions  : litellm {', '.join(h['versions'])}")
        print(f"  URL       : {h['job_url']}")
        print(f"  Log lines :")
        for line in h["context"]:
            print(f"    {line}")


if __name__ == "__main__":
    main()

export GITLAB_TOKEN="your-gitlab-pat"

python find_litellm_jobs.py

#!/usr/bin/env python3
"""
Scan all GitLab CI/CD jobs in a GitLab group that ran between
0800-1244 UTC today and identify any that installed litellm 1.82.7 or 1.82.8.

Adjust WINDOW_START / WINDOW_END to cover March 24, 2026 if running later.
"""

import os
import re
import sys
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import datetime, timezone

import requests

GITLAB_URL = "https://gitlab.com"
GROUP_NAME = "YourGroup"  # <-- set to your GitLab group name
TOKEN = os.environ.get("GITLAB_TOKEN", "")

TODAY = datetime.now(timezone.utc).date()
WINDOW_START = datetime(TODAY.year, TODAY.month, TODAY.day, 8, 0, 0, tzinfo=timezone.utc)
WINDOW_END   = datetime(TODAY.year, TODAY.month, TODAY.day, 12, 44, 0, tzinfo=timezone.utc)

TARGET_VERSIONS = {"1.82.7", "1.82.8"}
VERSION_PATTERN = re.compile(r"litellm[=\-](\d+\.\d+\.\d+)", re.IGNORECASE)

HEADERS = {"PRIVATE-TOKEN": TOKEN}
SESSION = requests.Session()
SESSION.headers.update(HEADERS)


def get_paginated(url, params=None):
    params = dict(params or {})
    params.setdefault("per_page", 100)
    page = 1
    while True:
        params["page"] = page
        resp = SESSION.get(url, params=params, timeout=30)
        resp.raise_for_status()
        data = resp.json()
        if not data:
            break
        yield from data
        if len(data) < params["per_page"]:
            break
        page += 1


def get_group_id(group_name):
    resp = SESSION.get(f"{GITLAB_URL}/api/v4/groups/{group_name}", timeout=30)
    resp.raise_for_status()
    return resp.json()["id"]


def get_all_projects(group_id):
    projects = []
    for p in get_paginated(
        f"{GITLAB_URL}/api/v4/groups/{group_id}/projects",
        {"include_subgroups": "true", "archived": "false"},
    ):
        projects.append({"id": p["id"], "name": p["path_with_namespace"]})
    return projects


def parse_ts(ts_str):
    if not ts_str:
        return None
    ts_str = ts_str.replace("Z", "+00:00")
    return datetime.fromisoformat(ts_str)


def jobs_in_window(project_id):
    matching = []
    url = f"{GITLAB_URL}/api/v4/projects/{project_id}/jobs"
    params = {"per_page": 100, "scope[]": ["success", "failed", "canceled", "running"]}

    page = 1
    while True:
        params["page"] = page
        resp = SESSION.get(url, params=params, timeout=30)
        if resp.status_code == 403:
            return matching
        resp.raise_for_status()
        jobs = resp.json()
        if not jobs:
            break

        stop_early = False
        for job in jobs:
            ts = parse_ts(job.get("started_at") or job.get("created_at"))
            if ts is None:
                continue
            if ts > WINDOW_END:
                continue
            if ts < WINDOW_START:
                stop_early = True
                continue
            matching.append(job)

        if stop_early or len(jobs) < 100:
            break
        page += 1

    return matching


def fetch_trace(project_id, job_id):
    url = f"{GITLAB_URL}/api/v4/projects/{project_id}/jobs/{job_id}/trace"
    resp = SESSION.get(url, timeout=60)
    if resp.status_code in (403, 404):
        return ""
    resp.raise_for_status()
    return resp.text


def check_job(project_name, project_id, job):
    job_id   = job["id"]
    job_name = job["name"]
    ref      = job.get("ref", "")
    started  = job.get("started_at", job.get("created_at", ""))

    trace = fetch_trace(project_id, job_id)
    if not trace:
        return None

    found_versions = set()
    for match in VERSION_PATTERN.finditer(trace):
        ver = match.group(1)
        if ver in TARGET_VERSIONS:
            found_versions.add(ver)

    if not found_versions:
        return None

    context_lines = []
    for line in trace.splitlines():
        if VERSION_PATTERN.search(line):
            ver_match = VERSION_PATTERN.search(line)
            if ver_match and ver_match.group(1) in TARGET_VERSIONS:
                context_lines.append(line.strip())

    return {
        "project":    project_name,
        "project_id": project_id,
        "job_id":     job_id,
        "job_name":   job_name,
        "ref":        ref,
        "started_at": started,
        "versions":   sorted(found_versions),
        "context":    context_lines[:10],
        "job_url":    f"{GITLAB_URL}/{project_name}/-/jobs/{job_id}",
    }


def main():
    if not TOKEN:
        print("ERROR: Set GITLAB_TOKEN environment variable.", file=sys.stderr)
        sys.exit(1)

    print(f"Time window : {WINDOW_START.isoformat()} -> {WINDOW_END.isoformat()}")
    print(f"Hunting for : litellm {', '.join(sorted(TARGET_VERSIONS))}")
    print()

    print(f"Resolving group '{GROUP_NAME}'...")
    group_id = get_group_id(GROUP_NAME)

    print("Fetching projects...")
    projects = get_all_projects(group_id)
    print(f"  Found {len(projects)} projects")
    print()

    all_jobs_to_check = []

    print("Scanning job listings for time window...")
    for proj in projects:
        try:
            jobs = jobs_in_window(proj["id"])
        except requests.HTTPError as e:
            print(f"  WARN: {proj['name']} - {e}", file=sys.stderr)
            continue
        if jobs:
            print(f"  {proj['name']}: {len(jobs)} job(s) in window")
        for j in jobs:
            all_jobs_to_check.append((proj["name"], proj["id"], j))

    total = len(all_jobs_to_check)
    print(f"\nFetching traces for {total} job(s)...")
    print()

    hits = []
    with ThreadPoolExecutor(max_workers=10) as pool:
        futures = {
            pool.submit(check_job, pname, pid, job): (pname, job["id"])
            for pname, pid, job in all_jobs_to_check
        }
        done = 0
        for future in as_completed(futures):
            done += 1
            pname, jid = futures[future]
            try:
                result = future.result()
            except Exception as e:
                print(f"  ERROR checking {pname} job {jid}: {e}", file=sys.stderr)
                continue
            if result:
                hits.append(result)
            print(f"  [{done}/{total}] checked {pname} job {jid}" +
                  (f"  *** HIT: litellm {result['versions']} ***" if result else ""),
                  flush=True)

    print()
    print("=" * 72)
    print(f"RESULTS: {len(hits)} job(s) installed litellm {' or '.join(sorted(TARGET_VERSIONS))}")
    print("=" * 72)

    if not hits:
        print("No matches found.")
        return

    for h in sorted(hits, key=lambda x: x["started_at"]):
        print()
        print(f"  Project   : {h['project']}")
        print(f"  Job       : {h['job_name']} (#{h['job_id']})")
        print(f"  Branch/tag: {h['ref']}")
        print(f"  Started   : {h['started_at']}")
        print(f"  Versions  : litellm {', '.join(h['versions'])}")
        print(f"  URL       : {h['job_url']}")
        print(f"  Log lines :")
        for line in h["context"]:
            print(f"    {line}")


if __name__ == "__main__":
    main()