docs: document LITELLM_MODEL_COST_MAP_URL for firewalled environments by moshemorad · Pull Request #2054 · HolmesGPT/holmesgpt

moshemorad · 2026-05-17T12:36:10Z

Summary

Documents LITELLM_MODEL_COST_MAP_URL and Robusta's mirror of LiteLLM's model catalog (model_prices_and_context_window.json).

Customers whose egress firewalls block raw.githubusercontent.com cannot let LiteLLM refresh its model catalog (which determines per-model context windows, max output tokens, and pricing). The fix is purely operational — LiteLLM already honors LITELLM_MODEL_COST_MAP_URL, and Robusta now serves a mirror of the file at https://api.robusta.dev/litellm/model_prices_and_context_window.json with TTL caching and a stale fallback. Setting the env var via additionalEnvVars in Helm is all it takes.

For fully self-hosted Robusta installs where the relay itself also cannot reach GitHub, the relay's LITELLM_MODEL_COST_MAP_UPSTREAM_URL can be pointed at Robusta's mirror to chain the lookup — documented inline.

Relay-side endpoint: robusta-dev/relay#533 (ROB-3898).

Test plan

Render docs/reference/environment-variables.md locally and confirm the new section renders correctly
Verify the linked relay endpoint returns valid JSON once feat(runbook): llm select the correct runbook from catalog #533 is merged and deployed
Confirm LITELLM_MODEL_COST_MAP_URL=https://api.robusta.dev/litellm/model_prices_and_context_window.json works end-to-end from a HolmesGPT pod that cannot reach raw.githubusercontent.com

Generated by Claude Code

Summary by CodeRabbit

Documentation
- Added guidance for a new configuration variable to override the default LiteLLM model cost map URL.
- Documented using an alternative mirror (with caching/fallback behavior) when direct downloads are restricted.
- Included a Helm example showing how to set the configuration for deployments.

Customers behind firewalls that block raw.githubusercontent.com cannot let LiteLLM refresh its model catalog. Robusta now mirrors the file at https://api.robusta.dev/litellm/model_prices_and_context_window.json, and LiteLLM honors LITELLM_MODEL_COST_MAP_URL out of the box. Documents the env var, the mirror URL, and the chained-relay setup for fully self-hosted deployments where the relay itself also cannot reach GitHub. Relay-side endpoint: robusta-dev/relay#533 (ROB-3898). Signed-off-by: Claude <noreply@anthropic.com>

claude

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

_{Tip: disable this comment in your organization's Code Review settings.}

coderabbitai · 2026-05-17T12:36:29Z

Walkthrough

Adds documentation for the LITELLM_MODEL_COST_MAP_URL environment variable, documents the Robusta mirror URL, and provides a Helm additionalEnvVars example for setting the variable.

Changes

LiteLLM Model Cost Map Configuration Documentation

Layer / File(s)	Summary
LITELLM_MODEL_COST_MAP_URL environment variable documentation `docs/reference/environment-variables.md`	Adds documentation for the `LITELLM_MODEL_COST_MAP_URL` environment variable, shows the Robusta mirror URL `https://api.robusta.dev/litellm/model_prices_and_context_window.json`, and provides a Helm `additionalEnvVars` snippet demonstrating how to set the variable.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Suggested reviewers

arikalon1

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically describes the main change: documenting a new environment variable for firewalled environments, which directly matches the PR's core objective.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

netlify · 2026-05-17T12:36:48Z

✅ Deploy Preview for holmes-docs ready!

Name	Link
🔨 Latest commit	`f79c82b`
🔍 Latest deploy log	https://app.netlify.com/projects/holmes-docs/deploys/6a0d6b1122737e0008b2eb56
😎 Deploy Preview	https://deploy-preview-2054--holmes-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/reference/environment-variables.md`:
- Around line 188-190: The fenced code block containing the URL snippet
currently lacks a language identifier (MD040); update the triple-backtick fence
for the block that contains
"https://api.robusta.dev/litellm/model_prices_and_context_window.json" to
include a language token (for example add "text" after the opening ```), so the
block becomes a proper fenced code block with a language specifier for correct
rendering.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3daf8874-2b6e-49b8-b4cb-536695c58ba6

📥 Commits

Reviewing files that changed from the base of the PR and between fa5e0ce and a60b30b.

📒 Files selected for processing (1)

docs/reference/environment-variables.md

…ction The self-hosted-relay chaining note adds noise for the common case (a SaaS customer pointing at api.robusta.dev) without giving fully self-hosted users enough detail to act on. Keep the section focused on the env var and the mirror URL. Signed-off-by: Claude <noreply@anthropic.com>

coderabbitai

♻️ Duplicate comments (1)

docs/reference/environment-variables.md (1)
188-190: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add a language identifier to the fenced URL block.

This still triggers MD040 and should use a language token (e.g., text) for consistent markdown rendering/linting.
Proposed fix
-```
+```text
 https://api.robusta.dev/litellm/model_prices_and_context_window.json
</details>

<details>
<summary>🤖 Prompt for AI Agents</summary>
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @docs/reference/environment-variables.md around lines 188 - 190, The fenced
code block containing the URL should include a language identifier to satisfy
MD040; update the block that currently shows the URL
"https://api.robusta.dev/litellm/model_prices_and_context_window.json" by
changing the opening fence from totext so the block is text ... ,
ensuring consistent markdown linting and rendering.
</details>

</blockquote></details>

</blockquote></details>

<details>
<summary>🤖 Prompt for all review comments with AI agents</summary>
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In @docs/reference/environment-variables.md:

Around line 188-190: The fenced code block containing the URL should include a
language identifier to satisfy MD040; update the block that currently shows the
URL "https://api.robusta.dev/litellm/model_prices_and_context_window.json" by
changing the opening fence from totext so the block is text ... ,
ensuring consistent markdown linting and rendering.
</details>

---

<details>
<summary>ℹ️ Review info</summary>

<details>
<summary>⚙️ Run configuration</summary>

**Configuration used**: Organization UI

**Review profile**: CHILL

**Plan**: Pro

**Run ID**: `3b05c455-0204-43e2-a726-8c81b8e4c7d7`

</details>

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between a60b30b989719d57c778c27a562bb4ca1c7e6fe1 and c8b6bd6ce40eee00fe81a13270405d6407e9eca5.

</details>

<details>
<summary>📒 Files selected for processing (1)</summary>

* `docs/reference/environment-variables.md`

</details>

</details>

claude Bot reviewed May 17, 2026

View reviewed changes

coderabbitai Bot reviewed May 17, 2026

View reviewed changes

Comment thread docs/reference/environment-variables.md

coderabbitai Bot reviewed May 17, 2026

View reviewed changes

Merge branch 'master' into claude/fix-rob-3898-oKCKG

8c62764

RoiGlinik approved these changes May 20, 2026

View reviewed changes

Merge branch 'master' into claude/fix-rob-3898-oKCKG

f79c82b

RoiGlinik enabled auto-merge (squash) May 20, 2026 08:04

RoiGlinik merged commit d71f67a into master May 20, 2026
10 of 14 checks passed

RoiGlinik deleted the claude/fix-rob-3898-oKCKG branch May 20, 2026 08:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: document LITELLM_MODEL_COST_MAP_URL for firewalled environments#2054

docs: document LITELLM_MODEL_COST_MAP_URL for firewalled environments#2054
RoiGlinik merged 4 commits into
masterfrom
claude/fix-rob-3898-oKCKG

moshemorad commented May 17, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

claude Bot left a comment

Uh oh!

coderabbitai Bot commented May 17, 2026 •

edited

Loading

Uh oh!

netlify Bot commented May 17, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

moshemorad commented May 17, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Summary by CodeRabbit

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

coderabbitai Bot commented May 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Suggested reviewers

Uh oh!

netlify Bot commented May 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for holmes-docs ready!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

moshemorad commented May 17, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 17, 2026 •

edited

Loading

netlify Bot commented May 17, 2026 •

edited

Loading