Skip to content

fix(proxy): retry passthrough on transient upstream connection close - #1513

Merged
JerrettDavis merged 2 commits into
headroomlabs-ai:mainfrom
gaurav0107:fix/1112-proxy-returns-502-with-httpx-incomplete
Jul 6, 2026
Merged

JerrettDavis merged 2 commits into
headroomlabs-ai:mainfrom
gaurav0107:fix/1112-proxy-returns-502-with-httpx-incomplete

Conversation

@gaurav0107

@gaurav0107 gaurav0107 commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Description

GET /v1/models (and other buffered passthrough routes) returned an opaque
HTTP 502 when an OpenAI-compatible upstream closed a pooled keep-alive
connection mid-response, surfacing
httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read). The same upstream answers a direct
curl with 200 because curl opens a fresh connection per call, while Headroom
reuses pooled keep-alive connections — so the first request issued on a stale
connection fails even though the upstream is healthy.

The fix makes the buffered passthrough path retry once on a fresh connection
(exactly what curl does), and return a clear error only if the upstream is
genuinely sending an incomplete response.

Closes #1112

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Performance improvement
  • Code refactoring (no functional changes)

Changes Made

  • Add headroom.proxy.helpers.request_with_transient_retry(client, *, request_id=None, max_retries=1, **request_kwargs): issues a buffered httpx request and retries on a fresh connection when (and only when) httpx.RemoteProtocolError is raised. Every other exception (ConnectError, timeouts, status errors) propagates immediately, so existing handling is unchanged. Documented as buffered-only (a streamed response can't be safely replayed once bytes reach the client).
  • Route OpenAIHandlerMixin.handle_passthrough through the helper, and add an except httpx.RemoteProtocolError arm that returns a clear 502 with error type upstream_protocol_error when the protocol error persists across the retry (instead of letting the raw error surface as an opaque/unhandled 502).
  • Add tests/test_proxy_passthrough_transient_retry.py (helper unit tests + handler-level tests covering the exact issue path).
  • Add a CHANGELOG.md entry under Unreleased → Fixed.

Scope note: streaming /v1/responses is intentionally out of scope for this
change — a streamed response cannot be safely retried after the first byte has
been delivered to the client. The helper is written reusable so a
streaming-aware follow-up can build on it.

Testing

  • Unit tests pass (pytest)
  • Linting passes (ruff check .)
  • Type checking passes (mypy headroom)
  • New tests added for new functionality
  • Manual testing performed

Test Output

$ ruff check headroom/proxy/helpers.py headroom/proxy/handlers/openai.py tests/test_proxy_passthrough_transient_retry.py
All checks passed!

$ mypy headroom/proxy/helpers.py --ignore-missing-imports
Success: no issues found in 1 source file

$ pytest tests/test_proxy_passthrough_transient_retry.py -q
tests/test_proxy_passthrough_transient_retry.py .......                  [100%]
7 passed in 0.27s

# no regressions in the surrounding passthrough/handler suites:
$ pytest tests/test_proxy_passthrough_transient_retry.py tests/test_proxy_handler_helpers.py \
         tests/test_proxy_byte_faithful_forwarding.py \
         tests/test_proxy/test_compression_failure_action.py tests/test_proxy_copilot_auth_hooks.py -q
80 passed, 1 warning in 6.88s

Real Behavior Proof

Reproduced against a real local TCP server (no mocks) that speaks HTTP/1.1
and, when armed, emits a chunked body then closes the socket without the
terminating 0\r\n\r\n — the exact condition that makes httpx raise the
incomplete chunked read error from this issue.

  • Environment: macOS arm64, Python 3.12, httpx 0.28.1 (same httpx major as the report), real loopback sockets via asyncio.start_server.
  • Exact command / steps: start the local server; (1) issue a single buffered request — the pre-fix handle_passthrough behaviour; (2) issue the same request through request_with_transient_retry — the fix. Verbatim: python repro_1112.py.
  • Observed result: BEFORE the fix a single request raises httpx.RemoteProtocolError ("incomplete chunked read") which handle_passthrough surfaced as an opaque HTTP 502; AFTER the fix the same request returns HTTP 200 (the retry opened a fresh connection, mirroring a direct curl). Full terminal output:
upstream listening on http://127.0.0.1:62374/v1/models

BEFORE (single buffered request, pre-fix behaviour):
  raised httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)
  -> handle_passthrough surfaced this as an opaque HTTP 502

AFTER (request_with_transient_retry, the fix):
  HTTP 200  body={"object":"list","data":[]}
  -> first attempt hit the incomplete chunked read, retry on a
     fresh connection returned 200 (mirrors a direct curl)

The log line Upstream closed connection mid-response (...incomplete chunked read); retrying on a fresh connection (attempt 1/1) fires on the recovered
request, confirming the retry path is what produced the 200.

  • Not tested: real third-party upstreams (LiteLLM/vLLM/etc.) — the local server reproduces the precise httpx error deterministically; the streaming /v1/responses path is intentionally out of scope (a streamed response cannot be safely retried after the first byte reaches the client).

Review Readiness

  • I have performed a self-review
  • This PR is ready for human review

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have updated the CHANGELOG.md if applicable

Additional Notes

  • No new dependencies (httpx is already a proxy dependency), so no supply-chain justification is required.
  • The retry is deliberately narrow: only httpx.RemoteProtocolError is retried, capped at one retry, so a genuinely-down upstream still fails fast via the existing ConnectError/timeout path.
  • "Documentation" checklist item refers to the CHANGELOG.md entry; no user-facing docs pages needed for this internal resilience fix.

@github-actions

github-actions Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

PR governance

This PR follows the template and is marked ready for human review.

@github-actions github-actions Bot added the status: needs author action Pull request body or readiness checklist still needs author updates label Jun 27, 2026
@gaurav0107
gaurav0107 marked this pull request as ready for review June 27, 2026 23:13
@github-actions github-actions Bot added status: ready for review Pull request body is complete and the author marked it ready for human review and removed status: needs author action Pull request body or readiness checklist still needs author updates labels Jun 27, 2026

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The retry path looks narrowly scoped and matches the failure mode from #1112: only RemoteProtocolError is retried, the second failure is surfaced as a clean 502 instead of bubbling an exception, and the handler tests cover both transient recovery and persistent upstream close. No code changes requested.

Comment thread headroom/proxy/handlers/openai.py Fixed
Comment thread headroom/proxy/handlers/openai.py Fixed

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the latest CodeQL follow-up. The client-facing 502 message is now generic while the detailed upstream protocol exception remains in the server-side warning log, so this preserves operator diagnosability without leaking upstream exception text to callers. No additional code changes requested.

This branch still needs to be updated from main: GitHub marks it conflicting, and the remaining pip-audit failure is the stale transformers CVE already fixed on main.

GET /v1/models and other buffered passthrough routes returned an opaque
HTTP 502 when an OpenAI-compatible upstream closed a pooled keep-alive
connection mid-response, surfacing httpx.RemoteProtocolError ("peer closed
connection without sending complete message body (incomplete chunked read)").
A direct curl works because it opens a fresh connection per call; Headroom
reuses pooled connections, so a stale connection fails on reuse even though
the upstream is healthy.

Add headroom.proxy.helpers.request_with_transient_retry, which retries a
buffered request once on a fresh connection (only on RemoteProtocolError, so
ConnectError/timeout handling is unchanged), and route handle_passthrough
through it with a clear upstream_protocol_error 502 when the error persists.

Closes headroomlabs-ai#1112
The RemoteProtocolError handler added for the headroomlabs-ai#1112 transient-retry fix
interpolated the raw httpx exception into the JSON error message returned
to the external client, so upstream stack-trace/exception text could leak
to callers (CodeQL py/stack-trace-exposure, alert headroomlabs-ai#136 at openai.py:6341).

Keep the full exception in the server-side logger.warning (unchanged) and
return a generic "upstream closed the connection without sending a complete
response" message with the same 502 status. Only the flagged path is
touched; behaviour and the existing 502 contract are otherwise unchanged.
@gaurav0107
gaurav0107 force-pushed the fix/1112-proxy-returns-502-with-httpx-incomplete branch from 1cdcf8a to 1e1b6b3 Compare July 2, 2026 15:14
@gaurav0107

Copy link
Copy Markdown
Contributor Author

Rebased onto the latest main to clear the merge conflict (the branch had fallen 8 commits behind). Two conflicts, both resolved by keeping both sides:

No logic changes: the approved retry fix and the CodeQL sanitization (generic client-facing 502, full detail stays in the server-side log) are unchanged. Force-pushed 1e1b6b3; the PR now shows mergeable.

Local checks on the affected area are green: ruff check, ruff format --check, mypy (helpers.py + handlers/openai.py), and the passthrough/handler suites (test_proxy_passthrough_transient_retry, test_proxy_handler_helpers, test_proxy_byte_faithful_forwarding, test_proxy/test_compression_failure_action, test_proxy_copilot_auth_hooks) — 84 passed. The re-run CI workflows are currently waiting on maintainer approval.

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the rebased head. The merge kept the retry helper alongside the newer overload-status helper, and the CodeQL follow-up still keeps detailed upstream exception text in the server log while returning a generic upstream_protocol_error message to clients. I do not see a remaining code blocker.

Only governance checks are attached to this latest head while the substantive workflows wait for permission, so this approval is based on review of the final diff and the reported focused local checks rather than new GitHub CI signal.

@JerrettDavis
JerrettDavis merged commit 5d14080 into headroomlabs-ai:main Jul 6, 2026
27 checks passed
@github-actions github-actions Bot mentioned this pull request Jul 9, 2026
chopratejas pushed a commit that referenced this pull request Jul 9, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>0.31.0</summary>

##
[0.31.0](v0.30.0...v0.31.0)
(2026-07-09)


### Features

* **cache:** provider-agnostic cache-mode delta + cc-agnostic prefix
comparison
([#1868](#1868))
([7c2f0ea](7c2f0ea))
* **ccr:** wire retrieve-tool interception into OpenAI Responses handler
([#1898](#1898))
([62cd307](62cd307))
* **compression:** add audit-safe mode with protected pattern matching
([#1899](#1899))
([bb112dd](bb112dd))
* **content-router:** accept any real compression (remove min-savings
floor)
([#1771](#1771))
([6c31db9](6c31db9))
* **content-router:** lossless-first dispatch, cross-turn dedup, and A7
lossy-after-fold
([#1818](#1818))
([60af15f](60af15f))
* **proxy:** add provider-only HTTP proxy
([#1807](#1807))
([ebe0a3b](ebe0a3b))
* **proxy:** add turn-hook extension point for buffered model turns
([#1891](#1891))
([ec950f7](ec950f7))


### Bug Fixes

* **build:** enable Intel macOS pip installs via ort-load-dynamic
([#1538](#1538))
([32ce99e](32ce99e))
* **cache:** avoid fallback session collisions
([#1827](#1827))
([0f606b6](0f606b6))
* **ccr:** make expired retrieve misses terminal
([#1781](#1781))
([9cbdba4](9cbdba4))
* **ccr:** preserve Anthropic re-stream shape
([#1854](#1854))
([f663894](f663894))
* **ccr:** preserve thinking blocks in buffered stream re-synthesis
([#1897](#1897))
([ede085c](ede085c))
* **cli/proxy:** preserve explicit HEADROOM_MIN_TOKENS=0 / MAX_ITEMS=0
([#1886](#1886))
([3a33af1](3a33af1))
* **code-compressor:** CJK-aware relevance-query symbol matching
([#1747](#1747))
([b38315c](b38315c))
* **codex:** discover updated Codex state stores
([#1889](#1889))
([9d42eba](9d42eba))
* **codex:** OpenCode Zen telemetry attribution
([#1648](#1648))
([f18c6bd](f18c6bd))
* **content-detector:** detect and compress space-separated JSON objects
([#1742](#1742))
([5194bdc](5194bdc))
* **content-router:** token-measure lossless folds at the acceptance
gate ([#1772](#1772))
([c5493ea](c5493ea))
* **copilot:** normalize subscription routing host
([#1836](#1836))
([afd9cbd](afd9cbd))
* **copilot:** route mixed-model requests per model
([#1785](#1785))
([5af5e22](5af5e22))
* **dashboard:** deduplicate repeated savings metrics
([#1804](#1804))
([88f935a](88f935a))
* **dashboard:** distinguish unavailable RTK from zero stats in Docker
([#1900](#1900))
([87f6e93](87f6e93))
* **dashboard:** distinguish unavailable RTK from zero stats in Docker
([#1901](#1901))
([361adcd](361adcd))
* **dashboard:** price proxy savings without litellm
([#1728](#1728))
([188e382](188e382))
* detect and clear stale ANTHROPIC_BASE_URL from crashed wrap sessions
([#1768](#1768))
([#1837](#1837))
([84509a4](84509a4))
* **docker:** persist headroom workspace in compose
([#1839](#1839))
([5e29c06](5e29c06))
* **docker:** report source build version
([#1862](#1862))
([3807488](3807488))
* **evals:** default unparseable judge scores below pass threshold
([#1892](#1892))
([42ebbc6](42ebbc6))
* **install:** pass sc.exe create as raw command line so binPath=
quoting survives
([#1654](#1654))
([#1702](#1702))
([d6e0710](d6e0710))
* **install:** persist --no-http2 override through install apply
([#1676](#1676))
([6fb5f3b](6fb5f3b))
* **mcp:** isolate ClaudeRegistrar CLI config env
([#1888](#1888))
([1c947b1](1c947b1))
* **mcp:** surface dead proxy state
([#1786](#1786))
([931eed8](931eed8))
* **memory:** resolve Trae cwd metadata from user reminders
([#1737](#1737))
([#1887](#1887))
([3e85eb1](3e85eb1))
* **opencode:** use local MCP config
([#1383](#1383))
([4bd3ddf](4bd3ddf))
* **proxy/openai:** thread savings-profile kwargs into chat completions
([#1606](#1606))
([7ff842d](7ff842d))
* **proxy/openai:** translate max_tokens -&gt; max_completion_tokens on
chat path
([#1774](#1774))
([285808b](285808b))
* **proxy:** bound Codex WS compression fallback latency
([#1802](#1802))
([d24a3f8](d24a3f8))
* **proxy:** bound HF tokenizer load and offload token counting off
event loop
([#1738](#1738))
([46d5d68](46d5d68))
* **proxy:** cancel retry backoff on shutdown
([#1834](#1834))
([da2d8dc](da2d8dc))
* **proxy:** compress Anthropic user text blocks when enabled
([#1875](#1875))
([e36439a](e36439a))
* **proxy:** freeze must forward cached (compressed) prefix
byte-identical — stop token-mode cache busting
([#1850](#1850))
([248ae0f](248ae0f))
* **proxy:** fsync savings dir after atomic rename
([#1764](#1764))
([7de2c1e](7de2c1e))
* **proxy:** keep cache_control bounded + stable so the freeze overlay
stops busting
([#1852](#1852))
([4820134](4820134))
* **proxy:** persist lifetime cache-read savings across restarts
([#1665](#1665))
([908997e](908997e))
* **proxy:** preserve streaming passthrough beta headers
([#1783](#1783))
([0f553a8](0f553a8))
* **proxy:** release _active_streams session lock on setup-phase errors
([#1864](#1864))
([2ccd831](2ccd831))
* **proxy:** retry HTTP/2 stream resets instead of 502ing
([#1645](#1645))
([2ce19c2](2ce19c2))
* **proxy:** retry passthrough on transient upstream connection close
([#1513](#1513))
([5d14080](5d14080))
* **proxy:** route Foundry Anthropic messages
([#1878](#1878))
([739f654](739f654))
* **proxy:** serve /favicon.ico locally instead of tunneling upstream
([#1787](#1787))
([#1847](#1847))
([3076e32](3076e32))
* **proxy:** stop rtk stat failures from corrupting session baseline
([#1693](#1693))
([681b9a8](681b9a8))
* **proxy:** strip 1m model suffix before upstream forwarding
([#1840](#1840))
([e22d745](e22d745))
* **proxy:** subtract cache write premiums from net savings
([#1800](#1800))
([53a465b](53a465b))
* **router:** honor MCP aliases in excluded tools
([#1822](#1822))
([#1863](#1863))
([140d6e4](140d6e4))
* **rtk:** link managed rtk onto PATH instead of mutating the hook
([#1698](#1698))
([140cb05](140cb05))
* **streaming:** preserve server_tool_use sse blocks
([#1826](#1826))
([4ac5493](4ac5493))
* **toin:** publish skip compression recommendations
([#1782](#1782))
([be51008](be51008))
* **transforms:** normalize diff compressor context
([#1801](#1801))
([838c523](838c523))
* **transforms:** pass through ragged tables instead of misaligning
columns
([#1713](#1713))
([c7665ca](c7665ca))
* use rtk native Cursor hook instead of injecting .cursorrules
([#756](#756))
([#1846](#1846))
([1573f1f](1573f1f))
* **wrap:** replace stale-proxy detection with Vite-style port fallback
([#1406](#1406))
([b4205c6](b4205c6))


### Performance Improvements

* **proxy:** cap compression workers to CPU count
([#1803](#1803))
([0a3851b](0a3851b))
* **savings:** batch tracker persistence off the request hot path
([#1817](#1817))
([451b9f0](451b9f0))


### Dependencies

* bump the cargo-minor-patch group across 1 directory with 7 updates
([#1909](#1909))
([45601d9](45601d9))
* bump the npm-minor-patch group across 4 directories with 18 updates
([#1907](#1907))
([8872bbc](8872bbc))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
JerrettDavis pushed a commit to peterlodri-sec/headroom that referenced this pull request Jul 14, 2026
…eadroomlabs-ai#1513)

## Description

`GET /v1/models` (and other buffered passthrough routes) returned an
opaque
HTTP **502** when an OpenAI-compatible upstream closed a pooled
keep-alive
connection mid-response, surfacing
`httpx.RemoteProtocolError: peer closed connection without sending
complete
message body (incomplete chunked read)`. The same upstream answers a
direct
`curl` with 200 because curl opens a fresh connection per call, while
Headroom
reuses pooled keep-alive connections — so the first request issued on a
stale
connection fails even though the upstream is healthy.

The fix makes the buffered passthrough path retry once on a fresh
connection
(exactly what curl does), and return a clear error only if the upstream
is
genuinely sending an incomplete response.

Closes headroomlabs-ai#1112

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `headroom.proxy.helpers.request_with_transient_retry(client, *,
request_id=None, max_retries=1, **request_kwargs)`: issues a buffered
httpx request and retries on a **fresh connection** when (and only when)
`httpx.RemoteProtocolError` is raised. Every other exception
(`ConnectError`, timeouts, status errors) propagates immediately, so
existing handling is unchanged. Documented as buffered-only (a streamed
response can't be safely replayed once bytes reach the client).
- Route `OpenAIHandlerMixin.handle_passthrough` through the helper, and
add an `except httpx.RemoteProtocolError` arm that returns a clear `502`
with error type `upstream_protocol_error` when the protocol error
persists across the retry (instead of letting the raw error surface as
an opaque/unhandled 502).
- Add `tests/test_proxy_passthrough_transient_retry.py` (helper unit
tests + handler-level tests covering the exact issue path).
- Add a `CHANGELOG.md` entry under `Unreleased → Fixed`.

Scope note: streaming `/v1/responses` is intentionally **out of scope**
for this
change — a streamed response cannot be safely retried after the first
byte has
been delivered to the client. The helper is written reusable so a
streaming-aware follow-up can build on it.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ ruff check headroom/proxy/helpers.py headroom/proxy/handlers/openai.py tests/test_proxy_passthrough_transient_retry.py
All checks passed!

$ mypy headroom/proxy/helpers.py --ignore-missing-imports
Success: no issues found in 1 source file

$ pytest tests/test_proxy_passthrough_transient_retry.py -q
tests/test_proxy_passthrough_transient_retry.py .......                  [100%]
7 passed in 0.27s

# no regressions in the surrounding passthrough/handler suites:
$ pytest tests/test_proxy_passthrough_transient_retry.py tests/test_proxy_handler_helpers.py \
         tests/test_proxy_byte_faithful_forwarding.py \
         tests/test_proxy/test_compression_failure_action.py tests/test_proxy_copilot_auth_hooks.py -q
80 passed, 1 warning in 6.88s
```

## Real Behavior Proof

Reproduced against a **real local TCP server** (no mocks) that speaks
HTTP/1.1
and, when armed, emits a chunked body then closes the socket **without**
the
terminating `0\r\n\r\n` — the exact condition that makes httpx raise the
`incomplete chunked read` error from this issue.

- Environment: macOS arm64, Python 3.12, httpx 0.28.1 (same httpx major
as the report), real loopback sockets via `asyncio.start_server`.
- Exact command / steps: start the local server; (1) issue a single
buffered request — the pre-fix `handle_passthrough` behaviour; (2) issue
the same request through `request_with_transient_retry` — the fix.
Verbatim: `python repro_1112.py`.
- Observed result: BEFORE the fix a single request raises
`httpx.RemoteProtocolError` ("incomplete chunked read") which
`handle_passthrough` surfaced as an opaque HTTP 502; AFTER the fix the
same request returns **HTTP 200** (the retry opened a fresh connection,
mirroring a direct `curl`). Full terminal output:

```text
upstream listening on http://127.0.0.1:62374/v1/models

BEFORE (single buffered request, pre-fix behaviour):
  raised httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)
  -> handle_passthrough surfaced this as an opaque HTTP 502

AFTER (request_with_transient_retry, the fix):
  HTTP 200  body={"object":"list","data":[]}
  -> first attempt hit the incomplete chunked read, retry on a
     fresh connection returned 200 (mirrors a direct curl)
```

The log line `Upstream closed connection mid-response (...incomplete
chunked
read); retrying on a fresh connection (attempt 1/1)` fires on the
recovered
request, confirming the retry path is what produced the 200.

- Not tested: real third-party upstreams (LiteLLM/vLLM/etc.) — the local
server reproduces the precise httpx error deterministically; the
streaming `/v1/responses` path is intentionally out of scope (a streamed
response cannot be safely retried after the first byte reaches the
client).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

- No new dependencies (httpx is already a proxy dependency), so no
supply-chain justification is required.
- The retry is deliberately narrow: only `httpx.RemoteProtocolError` is
retried, capped at one retry, so a genuinely-down upstream still fails
fast via the existing `ConnectError`/timeout path.
- "Documentation" checklist item refers to the `CHANGELOG.md` entry; no
user-facing docs pages needed for this internal resilience fix.
JerrettDavis pushed a commit to peterlodri-sec/headroom that referenced this pull request Jul 14, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>0.31.0</summary>

##
[0.31.0](headroomlabs-ai/headroom@v0.30.0...v0.31.0)
(2026-07-09)


### Features

* **cache:** provider-agnostic cache-mode delta + cc-agnostic prefix
comparison
([headroomlabs-ai#1868](headroomlabs-ai#1868))
([7c2f0ea](headroomlabs-ai@7c2f0ea))
* **ccr:** wire retrieve-tool interception into OpenAI Responses handler
([headroomlabs-ai#1898](headroomlabs-ai#1898))
([62cd307](headroomlabs-ai@62cd307))
* **compression:** add audit-safe mode with protected pattern matching
([headroomlabs-ai#1899](headroomlabs-ai#1899))
([bb112dd](headroomlabs-ai@bb112dd))
* **content-router:** accept any real compression (remove min-savings
floor)
([headroomlabs-ai#1771](headroomlabs-ai#1771))
([6c31db9](headroomlabs-ai@6c31db9))
* **content-router:** lossless-first dispatch, cross-turn dedup, and A7
lossy-after-fold
([headroomlabs-ai#1818](headroomlabs-ai#1818))
([60af15f](headroomlabs-ai@60af15f))
* **proxy:** add provider-only HTTP proxy
([headroomlabs-ai#1807](headroomlabs-ai#1807))
([ebe0a3b](headroomlabs-ai@ebe0a3b))
* **proxy:** add turn-hook extension point for buffered model turns
([headroomlabs-ai#1891](headroomlabs-ai#1891))
([ec950f7](headroomlabs-ai@ec950f7))


### Bug Fixes

* **build:** enable Intel macOS pip installs via ort-load-dynamic
([headroomlabs-ai#1538](headroomlabs-ai#1538))
([32ce99e](headroomlabs-ai@32ce99e))
* **cache:** avoid fallback session collisions
([headroomlabs-ai#1827](headroomlabs-ai#1827))
([0f606b6](headroomlabs-ai@0f606b6))
* **ccr:** make expired retrieve misses terminal
([headroomlabs-ai#1781](headroomlabs-ai#1781))
([9cbdba4](headroomlabs-ai@9cbdba4))
* **ccr:** preserve Anthropic re-stream shape
([headroomlabs-ai#1854](headroomlabs-ai#1854))
([f663894](headroomlabs-ai@f663894))
* **ccr:** preserve thinking blocks in buffered stream re-synthesis
([headroomlabs-ai#1897](headroomlabs-ai#1897))
([ede085c](headroomlabs-ai@ede085c))
* **cli/proxy:** preserve explicit HEADROOM_MIN_TOKENS=0 / MAX_ITEMS=0
([headroomlabs-ai#1886](headroomlabs-ai#1886))
([3a33af1](headroomlabs-ai@3a33af1))
* **code-compressor:** CJK-aware relevance-query symbol matching
([headroomlabs-ai#1747](headroomlabs-ai#1747))
([b38315c](headroomlabs-ai@b38315c))
* **codex:** discover updated Codex state stores
([headroomlabs-ai#1889](headroomlabs-ai#1889))
([9d42eba](headroomlabs-ai@9d42eba))
* **codex:** OpenCode Zen telemetry attribution
([headroomlabs-ai#1648](headroomlabs-ai#1648))
([f18c6bd](headroomlabs-ai@f18c6bd))
* **content-detector:** detect and compress space-separated JSON objects
([headroomlabs-ai#1742](headroomlabs-ai#1742))
([5194bdc](headroomlabs-ai@5194bdc))
* **content-router:** token-measure lossless folds at the acceptance
gate ([headroomlabs-ai#1772](headroomlabs-ai#1772))
([c5493ea](headroomlabs-ai@c5493ea))
* **copilot:** normalize subscription routing host
([headroomlabs-ai#1836](headroomlabs-ai#1836))
([afd9cbd](headroomlabs-ai@afd9cbd))
* **copilot:** route mixed-model requests per model
([headroomlabs-ai#1785](headroomlabs-ai#1785))
([5af5e22](headroomlabs-ai@5af5e22))
* **dashboard:** deduplicate repeated savings metrics
([headroomlabs-ai#1804](headroomlabs-ai#1804))
([88f935a](headroomlabs-ai@88f935a))
* **dashboard:** distinguish unavailable RTK from zero stats in Docker
([headroomlabs-ai#1900](headroomlabs-ai#1900))
([87f6e93](headroomlabs-ai@87f6e93))
* **dashboard:** distinguish unavailable RTK from zero stats in Docker
([headroomlabs-ai#1901](headroomlabs-ai#1901))
([361adcd](headroomlabs-ai@361adcd))
* **dashboard:** price proxy savings without litellm
([headroomlabs-ai#1728](headroomlabs-ai#1728))
([188e382](headroomlabs-ai@188e382))
* detect and clear stale ANTHROPIC_BASE_URL from crashed wrap sessions
([headroomlabs-ai#1768](headroomlabs-ai#1768))
([headroomlabs-ai#1837](headroomlabs-ai#1837))
([84509a4](headroomlabs-ai@84509a4))
* **docker:** persist headroom workspace in compose
([headroomlabs-ai#1839](headroomlabs-ai#1839))
([5e29c06](headroomlabs-ai@5e29c06))
* **docker:** report source build version
([headroomlabs-ai#1862](headroomlabs-ai#1862))
([3807488](headroomlabs-ai@3807488))
* **evals:** default unparseable judge scores below pass threshold
([headroomlabs-ai#1892](headroomlabs-ai#1892))
([42ebbc6](headroomlabs-ai@42ebbc6))
* **install:** pass sc.exe create as raw command line so binPath=
quoting survives
([headroomlabs-ai#1654](headroomlabs-ai#1654))
([headroomlabs-ai#1702](headroomlabs-ai#1702))
([d6e0710](headroomlabs-ai@d6e0710))
* **install:** persist --no-http2 override through install apply
([headroomlabs-ai#1676](headroomlabs-ai#1676))
([6fb5f3b](headroomlabs-ai@6fb5f3b))
* **mcp:** isolate ClaudeRegistrar CLI config env
([headroomlabs-ai#1888](headroomlabs-ai#1888))
([1c947b1](headroomlabs-ai@1c947b1))
* **mcp:** surface dead proxy state
([headroomlabs-ai#1786](headroomlabs-ai#1786))
([931eed8](headroomlabs-ai@931eed8))
* **memory:** resolve Trae cwd metadata from user reminders
([headroomlabs-ai#1737](headroomlabs-ai#1737))
([headroomlabs-ai#1887](headroomlabs-ai#1887))
([3e85eb1](headroomlabs-ai@3e85eb1))
* **opencode:** use local MCP config
([headroomlabs-ai#1383](headroomlabs-ai#1383))
([4bd3ddf](headroomlabs-ai@4bd3ddf))
* **proxy/openai:** thread savings-profile kwargs into chat completions
([headroomlabs-ai#1606](headroomlabs-ai#1606))
([7ff842d](headroomlabs-ai@7ff842d))
* **proxy/openai:** translate max_tokens -&gt; max_completion_tokens on
chat path
([headroomlabs-ai#1774](headroomlabs-ai#1774))
([285808b](headroomlabs-ai@285808b))
* **proxy:** bound Codex WS compression fallback latency
([headroomlabs-ai#1802](headroomlabs-ai#1802))
([d24a3f8](headroomlabs-ai@d24a3f8))
* **proxy:** bound HF tokenizer load and offload token counting off
event loop
([headroomlabs-ai#1738](headroomlabs-ai#1738))
([46d5d68](headroomlabs-ai@46d5d68))
* **proxy:** cancel retry backoff on shutdown
([headroomlabs-ai#1834](headroomlabs-ai#1834))
([da2d8dc](headroomlabs-ai@da2d8dc))
* **proxy:** compress Anthropic user text blocks when enabled
([headroomlabs-ai#1875](headroomlabs-ai#1875))
([e36439a](headroomlabs-ai@e36439a))
* **proxy:** freeze must forward cached (compressed) prefix
byte-identical — stop token-mode cache busting
([headroomlabs-ai#1850](headroomlabs-ai#1850))
([248ae0f](headroomlabs-ai@248ae0f))
* **proxy:** fsync savings dir after atomic rename
([headroomlabs-ai#1764](headroomlabs-ai#1764))
([7de2c1e](headroomlabs-ai@7de2c1e))
* **proxy:** keep cache_control bounded + stable so the freeze overlay
stops busting
([headroomlabs-ai#1852](headroomlabs-ai#1852))
([4820134](headroomlabs-ai@4820134))
* **proxy:** persist lifetime cache-read savings across restarts
([headroomlabs-ai#1665](headroomlabs-ai#1665))
([908997e](headroomlabs-ai@908997e))
* **proxy:** preserve streaming passthrough beta headers
([headroomlabs-ai#1783](headroomlabs-ai#1783))
([0f553a8](headroomlabs-ai@0f553a8))
* **proxy:** release _active_streams session lock on setup-phase errors
([headroomlabs-ai#1864](headroomlabs-ai#1864))
([2ccd831](headroomlabs-ai@2ccd831))
* **proxy:** retry HTTP/2 stream resets instead of 502ing
([headroomlabs-ai#1645](headroomlabs-ai#1645))
([2ce19c2](headroomlabs-ai@2ce19c2))
* **proxy:** retry passthrough on transient upstream connection close
([headroomlabs-ai#1513](headroomlabs-ai#1513))
([5d14080](headroomlabs-ai@5d14080))
* **proxy:** route Foundry Anthropic messages
([headroomlabs-ai#1878](headroomlabs-ai#1878))
([739f654](headroomlabs-ai@739f654))
* **proxy:** serve /favicon.ico locally instead of tunneling upstream
([headroomlabs-ai#1787](headroomlabs-ai#1787))
([headroomlabs-ai#1847](headroomlabs-ai#1847))
([3076e32](headroomlabs-ai@3076e32))
* **proxy:** stop rtk stat failures from corrupting session baseline
([headroomlabs-ai#1693](headroomlabs-ai#1693))
([681b9a8](headroomlabs-ai@681b9a8))
* **proxy:** strip 1m model suffix before upstream forwarding
([headroomlabs-ai#1840](headroomlabs-ai#1840))
([e22d745](headroomlabs-ai@e22d745))
* **proxy:** subtract cache write premiums from net savings
([headroomlabs-ai#1800](headroomlabs-ai#1800))
([53a465b](headroomlabs-ai@53a465b))
* **router:** honor MCP aliases in excluded tools
([headroomlabs-ai#1822](headroomlabs-ai#1822))
([headroomlabs-ai#1863](headroomlabs-ai#1863))
([140d6e4](headroomlabs-ai@140d6e4))
* **rtk:** link managed rtk onto PATH instead of mutating the hook
([headroomlabs-ai#1698](headroomlabs-ai#1698))
([140cb05](headroomlabs-ai@140cb05))
* **streaming:** preserve server_tool_use sse blocks
([headroomlabs-ai#1826](headroomlabs-ai#1826))
([4ac5493](headroomlabs-ai@4ac5493))
* **toin:** publish skip compression recommendations
([headroomlabs-ai#1782](headroomlabs-ai#1782))
([be51008](headroomlabs-ai@be51008))
* **transforms:** normalize diff compressor context
([headroomlabs-ai#1801](headroomlabs-ai#1801))
([838c523](headroomlabs-ai@838c523))
* **transforms:** pass through ragged tables instead of misaligning
columns
([headroomlabs-ai#1713](headroomlabs-ai#1713))
([c7665ca](headroomlabs-ai@c7665ca))
* use rtk native Cursor hook instead of injecting .cursorrules
([headroomlabs-ai#756](headroomlabs-ai#756))
([headroomlabs-ai#1846](headroomlabs-ai#1846))
([1573f1f](headroomlabs-ai@1573f1f))
* **wrap:** replace stale-proxy detection with Vite-style port fallback
([headroomlabs-ai#1406](headroomlabs-ai#1406))
([b4205c6](headroomlabs-ai@b4205c6))


### Performance Improvements

* **proxy:** cap compression workers to CPU count
([headroomlabs-ai#1803](headroomlabs-ai#1803))
([0a3851b](headroomlabs-ai@0a3851b))
* **savings:** batch tracker persistence off the request hot path
([headroomlabs-ai#1817](headroomlabs-ai#1817))
([451b9f0](headroomlabs-ai@451b9f0))


### Dependencies

* bump the cargo-minor-patch group across 1 directory with 7 updates
([headroomlabs-ai#1909](headroomlabs-ai#1909))
([45601d9](headroomlabs-ai@45601d9))
* bump the npm-minor-patch group across 4 directories with 18 updates
([headroomlabs-ai#1907](headroomlabs-ai#1907))
([8872bbc](headroomlabs-ai@8872bbc))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: ready for review Pull request body is complete and the author marked it ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Proxy returns 502 with httpx incomplete chunked read against OpenAI-compatible upstream

3 participants