Value this issue delivers
A customer whose policy is "Bedrock only, in-VPC" cannot run the agent inference path today: verified on 2026-09-18, no Bedrock route exists anywhere in the runtime, and the components that call an LLM speak the OpenAI chat-completions shape. The choices are to teach every caller a second SDK, or to put one OpenAI-compatible gateway next to the deployment and point the existing base URL at it. The second costs no Vexa code change and works for any provider that operator already has — Bedrock is simply the case that forced it. What is missing is that the gateway is not part of the deployment surface: an operator has to invent it, and there is nothing to point a customer at.
What this issue ships
- A
llm-shim service in deploy/compose/docker-compose.yml behind the llm-shim profile — absent from docker compose config --services unless the profile is named, bound to loopback only, third-party image pinned by tag (litellm/litellm:main-v1.97.0).
deploy/compose/llm-shim/: the component README as its contract, a default config.yaml, and two worked examples — config.local.example.yaml and config.bedrock.example.yaml.- Helm:
llmShim.enabled (default off) rendering a Deployment, Service (ClusterIP) and ConfigMap, with existingSecret reaching the provider credential through secretKeyRef and extraEnv for anything else; nothing renders when it is off.
deploy/compose/.env.example: host AWS variables passed through only when set, with the comment saying so.docs/docs/configuration.mdx § Agent inference gains "The llm-shim": purpose, the compose command, the Helm value, the env table, the Bedrock case, and the loopback/no-auth warning.
Validation contract
Machine-observable: docker compose config --services with and without --profile llm-shim (absent / present); helm template in both states — zero mentions by default, and with llmShim.enabled=true one Deployment, Service and ConfigMap, the secretKeyRef to the named secret, the extra env rendered, one more readiness probe than the default render; the compose wiring tests, helm lint both ways, and the fast static gates green. Caller-verified: docker compose --profile llm-shim up -d llm-shim against any OpenAI-compatible server with one chat completion through it, and a live Bedrock request in-VPC on an AWS account, reported here.
Out of scope: any change to how Vexa components choose or call a model; authentication on the shim itself (loopback in compose, ClusterIP in Helm is the boundary); shipping a provider credential; making the shim a default component.
Refs: business custody DmitriyG228/biz#208. Delivered by #1711.
Value this issue delivers
A customer whose policy is "Bedrock only, in-VPC" cannot run the agent inference path today: verified on 2026-09-18, no Bedrock route exists anywhere in the runtime, and the components that call an LLM speak the OpenAI chat-completions shape. The choices are to teach every caller a second SDK, or to put one OpenAI-compatible gateway next to the deployment and point the existing base URL at it. The second costs no Vexa code change and works for any provider that operator already has — Bedrock is simply the case that forced it. What is missing is that the gateway is not part of the deployment surface: an operator has to invent it, and there is nothing to point a customer at.
What this issue ships
llm-shimservice indeploy/compose/docker-compose.ymlbehind thellm-shimprofile — absent fromdocker compose config --servicesunless the profile is named, bound to loopback only, third-party image pinned by tag (litellm/litellm:main-v1.97.0).deploy/compose/llm-shim/: the component README as its contract, a defaultconfig.yaml, and two worked examples —config.local.example.yamlandconfig.bedrock.example.yaml.llmShim.enabled(default off) rendering a Deployment, Service (ClusterIP) and ConfigMap, withexistingSecretreaching the provider credential throughsecretKeyRefandextraEnvfor anything else; nothing renders when it is off.deploy/compose/.env.example: host AWS variables passed through only when set, with the comment saying so.docs/docs/configuration.mdx§ Agent inference gains "The llm-shim": purpose, the compose command, the Helm value, the env table, the Bedrock case, and the loopback/no-auth warning.Validation contract
Machine-observable:
docker compose config --serviceswith and without--profile llm-shim(absent / present);helm templatein both states — zero mentions by default, and withllmShim.enabled=trueone Deployment, Service and ConfigMap, thesecretKeyRefto the named secret, the extra env rendered, one more readiness probe than the default render; the compose wiring tests,helm lintboth ways, and the fast static gates green. Caller-verified:docker compose --profile llm-shim up -d llm-shimagainst any OpenAI-compatible server with one chat completion through it, and a live Bedrock request in-VPC on an AWS account, reported here.Out of scope: any change to how Vexa components choose or call a model; authentication on the shim itself (loopback in compose, ClusterIP in Helm is the boundary); shipping a provider credential; making the shim a default component.
Refs: business custody DmitriyG228/biz#208. Delivered by #1711.