feat(client, server): request compression plugin#1643
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Important
Request-compression feature is well structured, but the server-side decompression path has exploitable resource-exhaustion risk and a few correctness/robustness issues that should be addressed before merging.
Reviewed changes — PR #1643 adds client and server request-compression plugins, a shared isCompressibleContentType helper, wiring tests across transports, and upgrades @standardserver from ^0.0.32 to ^0.2.0.
- Adds
RequestCompressionLinkPluginin@orpc/client— compresses request bodies above a configurable threshold using Web StreamsCompressionStream. - Adds
RequestCompressionHandlerPluginin@orpc/server— decompresses incoming request bodies usingDecompressionStreamwhenContent-Encodingis set. - Adds
isCompressibleContentTypeto@orpc/shared— regex-based allow-list for compressible content types. - Adds new fetch-adapter
BodyCompressionHandlerPluginfor response compression — mirrors the existing response-compression behavior with threshold and filter support. - Adds documentation and cross-transport integration tests (CrossWS, Hono fetch, MessagePort, Node HTTP/WS).
\u1f6a0 Decompression-bomb vulnerability in server plugin
The server plugin decompresses request bodies with no maximum output size, chunk count, or time budget. A small, malicious compressed payload can expand many-thousand-fold and exhaust memory or starve the event loop before any user procedure runs.
Technical details
# Decompression-bomb vulnerability in server plugin
## Affected sites
- `packages/server/src/plugins/request-compression.ts:34-48` — `resolveBody` unconditionally pipes through `new DecompressionStream(encoding)`.
## Required outcome
- Decompression must be bounded so that a small compressed payload cannot expand to unbounded memory/CPU use.
- The bound should be user-configurable; the default should be large enough for legitimate oRPC payloads but small enough to prevent abuse.
- Exceeding the limit should fail the request cleanly (e.g. 413 Payload Too Large or a plugin-specific error) without crashing the process.
## Suggested approach
Wrap the `DecompressionStream` in a `TransformStream` that tracks cumulative bytes and aborts when a configured `maxDecompressedBytes` is exceeded.
```ts
const limit = this.maxDecompressedBytes
let received = 0
const limited = new TransformStream<Uint8Array, Uint8Array>({
transform(chunk, controller) {
received += chunk.byteLength
if (received > limit) {
controller.error(new Error('Decompressed body exceeds maximum size'))
return
}
controller.enqueue(chunk)
},
})
const decompressedStream = stream
.pipeThrough(new DecompressionStream(encoding))
.pipeThrough(limited)
```
Expose the limit as a constructor option on `RequestCompressionHandlerPluginOptions`.
## Open questions for the human
- What is the intended maximum oRPC request body size in the supported runtimes (Vercel, Cloudflare Workers, Node, Bun)? The answer sets a reasonable default.
- Should the plugin be disabled by default to avoid surprising resource behavior on existing handlers?\u26a0\ufe0f FormData compression short-circuits on first non-compressible part
In the client plugin, a single non-compressible blob inside a FormData aborts the size calculation entirely. A later large compressible field can therefore keep the whole body uncompressed.
Technical details
# FormData compression short-circuits on first non-compressible part
## Affected sites
- `packages/client/src/plugins/request-compression.ts:116-119` — `break` exits the loop when a non-compressible blob part is found.
## Required outcome
- Non-compressible parts should be skipped during size estimation; compressible parts that follow must still count toward the threshold.
## Suggested approach
Replace `break` with `continue` so the loop keeps summing remaining parts.\u26a0\ufe0f ReadableStream bodies are skipped when size is unknown
The client plugin docs say “If the request size cannot be determined, compression will still be applied”, but for ReadableStream bodies compression is gated on a known, finite content-length header. A stream without content-length is silently sent uncompressed.
Technical details
# ReadableStream bodies are skipped when size is unknown
## Affected sites
- `packages/client/src/plugins/request-compression.ts:54-78` — branch checks `Number(flattenStandardHeader(request.headers['content-length']))`.
- `apps/content/docs/plugins/request-compression.md:22-27` — threshold option docstring.
## Required outcome
- Either compress `ReadableStream` bodies when the size is unknown, or update the docs to accurately describe the behavior.
## Suggested approach
If the intent is to compress unknown-size streams, remove the `content-length` requirement while keeping `content-type`. If the intent is to avoid buffering/chunk overhead, update the docstring and add a note that `content-length` is required for stream compression.\u26a0\ufe0f Content-Encoding header parsing is too strict, and multiple encodings are dropped silently
The server plugin treats Content-Encoding as a single literal token. Valid HTTP values like gzip, deflate fail the whitelist and the compressed body is passed downstream unchanged, leading to parsing failures.
Technical details
# Content-Encoding header parsing is too strict
## Affected sites
- `packages/server/src/plugins/request-compression.ts:16-22` — `encoding` is built from `flattenStandardHeader(...)` and checked with `SUPPORTED_ENCODINGS.includes(encoding)`.
## Required outcome
- The plugin should either support RFC-defined comma-separated encoding lists, or explicitly reject unsupported/ compound encodings with a clear error.
- Chained encodings (`gzip, deflate`) should not silently fall through to higher-level parsers.
## Suggested approach
Parse the header as a comma-separated list and act on the *last* supported encoding, or return an explicit 415-style error when any step in the chain is unsupported.\u26a0\ufe0f Client plugin preserves arbitrary content-encoding values
If a pre-existing interceptor sets any content-encoding header, the client plugin skips compression and forwards the body unchanged. An arbitrary value like __CUSTOM__ is therefore treated as already compressed, and the server plugin later ignores it. While not critical on its own, this is brittle and could hide mismatches between client and server.
Technical details
# Client plugin preserves arbitrary content-encoding values
## Affected sites
- `packages/client/src/plugins/request-compression.ts:49-52` — checks `contentEncoding !== undefined` but does not validate the value.
## Required outcome
- Only skip compression when the existing encoding is one the plugin actually supports. An unsupported/unknown encoding should not silently pass through unchanged.
## Suggested approach
Validate `contentEncoding` against the supported encodings list. If it is not supported, either throw or fall through to the normal body handling and overwrite it with the configured encoding rather than forwarding it unchanged.\u2139\ufe0f Regex-based content-type filter may be expensive on untrusted input
The COMPRESSIBLE_CONTENT_TYPE_REGEX in @orpc/shared is evaluated against attacker-controlled Content-Type strings and contains nested alternations. It currently has no length cap, so extremely long content-type headers could consume significant CPU.
Technical details
# Regex-based content-type filter may be expensive
## Affected sites
- `packages/shared/src/http.ts:58` — `COMPRESSIBLE_CONTENT_TYPE_REGEX` is compiled once and run on every compressible-body check.
## Required outcome
- Mitigate CPU-exhaustion risk from long or pathological content-type values without changing results for valid inputs.
## Suggested approach
Add a maximum input length guard (e.g. reject strings longer than 2048 characters before matching) or pre-normalize by trimming to the first semicolon.\u2139\ufe0f Body size estimation uses UTF-16 length, not byte length
RequestCompressionLinkPlugin uses string.length * AVG_BYTES_PER_CHAR for JSON, URLSearchParams, and FormData string fields. For payloads near the threshold that contain many non-ASCII characters, this estimate can be materially wrong. In addition, stringifyJSON(request.body) is called before the threshold check, so a non-serializable body (e.g. containing BigInt or a circular structure) throws before deciding not to compress.
Technical details
# Body size estimation uses UTF-16 length
## Affected sites
- `packages/client/src/plugins/request-compression.ts:7-9` — `AVG_BYTES_PER_CHAR` constant.
- `packages/client/src/plugins/request-compression.ts:127` — FormData string field estimate.
- `packages/client/src/plugins/request-compression.ts:154` — URLSearchParams estimate.
- `packages/client/src/plugins/request-compression.ts:174` — JSON body estimate.
## Required outcome
- Size estimates should be close to the actual wire-byte count for threshold decisions.
- Non-serializable bodies should not throw; the plugin should just not compress them.
## Suggested approach
Use `new TextEncoder().encode(string).byteLength` where practical. Wrap the JSON serialization in a `try`/`catch` and fall back to `next()` on failure.\u2139\ufe0f CompressionStream/DecompressionStream are assumed globals
Both plugins assume the runtime provides CompressionStream and DecompressionStream. On older Node versions they will throw at runtime. This is acceptable if the package’s declared engines/peer range already guarantees these globals, but worth confirming since the @standardserver upgrade is part of this PR.
Kimi K2 (free via Pullfrog for OSS) ｜ 𝕏
More templates
@orpc/arktype
@orpc/bun
@orpc/client
@orpc/cloudflare
@orpc/contract
@orpc/experimental-effect
@orpc/evlog
@orpc/interop
@orpc/json-schema
@orpc/nest
@orpc/next
@orpc/openapi
@orpc/opentelemetry
@orpc/pino
@orpc/publisher
@orpc/ratelimit
@orpc/server
@orpc/shared
@orpc/tanstack-query
@orpc/valibot
@orpc/zod
commit: |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Apply compression for ReadableStream, Blob, and FormData bodies when size is unknown, matching the threshold option docs.
There was a problem hiding this comment.
Important
Four of the prior review findings are now fixed in the new commits, but three correctness/security issues remain unaddressed: unlimited server decompression, arbitrary content-encoding passthrough on the client, and inaccurate JSON body-size estimation that can throw before the threshold check.
Reviewed changes — this run focused on the four new commits since 2b79bb77 (head now 41ba51f): the server plugin now parses comma-separated Content-Encoding values and decompresses in reverse application order; the client plugin compresses unknown-size ReadableStream/Blob bodies when the content type is compressible; FormData compression scoring now treats non-compressible blob parts as negative weight rather than short-circuiting; and isCompressibleContentType caps input length before running its regex.
- Server plugin handles chained
Content-Encoding—packages/server/src/plugins/request-compression.tsreplaces single-token parsing withparseContentEncodingsand iterates over encodings in reverse order. Tests coverdeflate, gzipand correctly reject mixed supported/unsupported chains (gzip, br). - Client plugin compresses unknown-size streams and blobs —
packages/client/src/plugins/request-compression.tsnow compressesReadableStreamandBlobbodies when the size cannot be determined, provided the content type is compressible. New unit tests exerciseNaNSizeBloband stream bodies withoutcontent-length. - FormData scoring fixed — non-compressible blob parts now subtract from a running score instead of breaking the loop, so later compressible parts can still push the total over the threshold. New tests cover mixed compressible/non-compressible parts.
- Content-type regex length cap added —
packages/shared/src/http.tsnow rejectsContent-Typevalues longer than 1024 characters before runningCOMPRESSIBLE_CONTENT_TYPE_REGEX.
🚨 Server plugin still has no decompression-bomb limit
The server plugin pipes through DecompressionStream for each entry in encodings without any limit on cumulative output size, chunk count, or time budget. The new multi-encoding support actually increases the attack surface because a chain like gzip, deflate can be crafted to explode in multiple stages.
Technical details
# Server plugin still has no decompression-bomb limit
## Affected sites
- `packages/server/src/plugins/request-compression.ts:43-48` — the reverse-order decompression loop unconditionally wraps the stream in `DecompressionStream`.
- `RequestCompressionHandlerPlugin` has no constructor options at all, so there is no place for callers to configure a bound.
## Required outcome
- Decompression must be bounded so that a small compressed payload cannot expand to unbounded memory/CPU use.
- The bound should be user-configurable with a sensible default.
- Exceeding the limit should fail the request cleanly (e.g. a 413-equivalent or plugin-specific error) without crashing the process.
## Suggested approach
Add a `maxDecompressedBytes` option to `RequestCompressionHandlerPluginOptions` and insert a size-limiting `TransformStream` after the final `DecompressionStream`. The prior review already included a code sketch for this.
## Open questions for the human
- What is the intended maximum oRPC request body size in the supported runtimes? The answer sets a reasonable default.
- Should the plugin be disabled by default to avoid surprising resource behavior on existing handlers?⚠️ Client plugin still forwards any pre-set content-encoding unchanged
contentEncoding !== undefined treats every existing value as "already compressed" and returns the body unchanged, even if the value is unsupported. A value like __CUSTOM__ will be forwarded to the server and the server plugin will ignore it because it is not in the supported list.
Technical details
# Client plugin forwards arbitrary content-encoding values
## Affected sites
- `packages/client/src/plugins/request-compression.ts:49-52` — `contentEncoding !== undefined` short-circuits compression without validating the value.
## Required outcome
- Only skip compression when the existing encoding is one the plugin actually supports.
- An unsupported/unknown encoding should not silently pass through unchanged.
## Suggested approach
Validate `contentEncoding` against the supported encodings list. If it is supported, return `next()` unchanged. If not, fall through to normal compression (which overwrites the header with the configured encoding) rather than forwarding the raw value.⚠️ JSON body size estimation is still approximate and can throw before the threshold check
stringifyJSON(request.body) is called unconditionally for JSON-like bodies, and the size estimate uses string.length * AVG_BYTES_PER_CHAR. Non-ASCII payloads near the threshold are estimated inaccurately, and bodies containing BigInt or circular structures will throw before the plugin even decides whether compression is worthwhile.
Technical details
# JSON body size estimation is still approximate and can throw before the threshold check
## Affected sites
- `packages/client/src/plugins/request-compression.ts:7-9` — `AVG_BYTES_PER_CHAR` constant.
- `packages/client/src/plugins/request-compression.ts:131` — FormData string field estimate.
- `packages/client/src/plugins/request-compression.ts:158` — URLSearchParams estimate.
- `packages/client/src/plugins/request-compression.ts:178-179` — JSON body estimate.
## Required outcome
- Size estimates should be close to the actual wire-byte count for threshold decisions.
- Non-serializable bodies should not throw; the plugin should just not compress them.
## Suggested approach
Use `new TextEncoder().encode(string).byteLength` where practical. Wrap the JSON serialization in a `try`/`catch` and fall back to `next()` on failure.Kimi K2 (free via Pullfrog for OSS) ｜ 𝕏
No description provided.