Summary

Netty's HTTP/3 QPACK prefixed-integer decoder (QpackUtil.decodePrefixedInteger) never bounds the number of continuation bytes. A remote, unauthenticated peer can send an unterminated integer (a first byte with all prefix bits set followed by an endless run of 0x80 continuation bytes) on either QPACK unidirectional stream. The ByteToMessageDecoder never consumes the bytes (the decoder returns -1/"need more"), so the cumulator grows without bound, and each decode() re-scans the whole buffer (O(N²) CPU). Result: OutOfMemoryError + event-loop CPU starvation → denial of service. Reachable in every configuration.

Root Cause

QpackUtil.decodePrefixedInteger (QpackUtil.java:86-114) walks continuation bytes with do { ... } while ((next & 0x80) == 0x80) and returns -1 (:104-106) on buffer exhaustion, resetting the reader index. Every caller treats <0 as "need more bytes" and returns without consuming. There is NO continuation-byte-count cap — unlike HPACK's HpackDecoder.decodeULE128, which enforces shift == 56 (codec-http2/.../HpackDecoder.java). An endless 0x80 run never reaches a terminator (high bit clear), so the caller never consumes and the QPACK-stream ByteToMessageDecoder cumulator grows unbounded (no setCumulator/size cap). Each decode() invocation additionally rescans the accumulated buffer from readerIndex to writerIndex via in.getByte(idx++), making accumulation O(N²) in CPU.

Impact

Remote unauthenticated DoS: unbounded per-connection heap growth (OutOfMemoryError) plus O(N²) event-loop CPU consumption. Reachable in every config (no dynamic-table gate on the affected branches). CWE-770 / CWE-400 / CWE-834.

Proof of Concept

1. Establish an HTTP/3 connection to a default Netty server.
2. Open a client-initiated unidirectional QUIC stream; write stream-type varint 0x03 (QPACK decoder) — or 0x02 (encoder).
3. Write a first byte with all prefix bits set so first == nbits and the continuation loop is entered:
   • decoder-stream Section Acknowledgment (7-bit prefix): 0xFF
   • encoder-stream Set Dynamic Table Capacity (5-bit prefix, gate-free): 0x3F
4. Stream 0x80 bytes indefinitely: (next & 0x80) == 0x80 is always true → loop only exits at buffer exhaustion → returns -1 (QpackUtil.java:106) → caller returns WITHOUT consuming.
5. Cumulator grows unbounded; each decode() rescans O(N²) → OutOfMemoryError + CPU starvation → DoS.

Note: a first byte of 0x80 is NOT a trigger (0x80 & 0x7F = 0 < 127 → returns immediately as a valid 1-byte value). The trigger requires the prefix bits all set (0xFF for 7-bit, 0x3F for 5-bit).

Attack Chain

1. Entry: HTTP/3 unidirectional stream, type 0x03 (or 0x02) → QpackDecoderHandler/QpackEncoderHandler installed. Guard: stream-type routing, one stream per type. Bypass proof: one stream suffices; unauthenticated.
2. Reach sink with no gate: QpackDecoderHandler Section-Ack (:57, decodePrefixedInteger(in,7)), Stream-Cancellation (:78, ,6), Insert-Count-Increment (:99, ,6) call decodePrefixedInteger on the first byte; QpackEncoderHandler Set-Dynamic-Table-Capacity (:68, ,5) is BEFORE the line-84 dynamic-table gate. Bypass proof: none of these branches is gated — reachable in every config, including assertions-on + dynamic-table-disabled.
3. Check (continuation loop): do { next = in.getByte(idx++) ... } while ((next & 0x80) == 0x80). Bypass proof: with next = 0x80 every iteration the condition is always true; the loop only exits at buffer exhaustion → return -1 (:106). No shift==56/factor bound exists in QpackUtil (only factor += 7).
4. Sink (caller): e.g. QpackDecoderHandler.java:58-60 sees <0 → returns without consuming → MERGE cumulator retains all bytes; each subsequent decode() rescans from readerIndex (O(N²)).
5. Impact: unbounded cumulator + O(N²) event-loop CPU → OutOfMemoryError / CPU starvation → DoS.

Bypass Evidence

• Trigger byte verified: (0x80 & 0x7F) = 0 returns early (not a trigger); (0xFF & 0x7F) = 127 = nbits enters the continuation loop; the 0x80 run never terminates.
• No continuation-count cap in QpackUtil (only factor += 7, no shift == 56 guard) — contrast HPACK HpackDecoder.decodeULE128 which HAS the guard.
• No setCumulator in the http3 module → default MERGE cumulator, no size cap.
• QUIC flow control does NOT bound accumulation: QuicheQuicStreamChannel.java:656-661 drains quiche's bounded buffer into the unbounded cumulator, then replenishes the window; readIfNoAutoRead (Http3CodecUtils.java:298) forces reads regardless of AUTO_READ.

Affected Versions

io.netty:netty-codec-http3 <= 4.2.16.Final (latest release; code present and unfixed on the tag, no post-tag commits to QPACK files). Distinct sink from GHSA-hpcc-26xq-25fv, GHSA-4grm-h2qv-h6w6, GHSA-c2rx-5r8w-8xr2.

Suggested Fix

Add a continuation-byte-count cap in QpackUtil.decodePrefixedInteger (port HPACK's shift == 56 overflow guard) so an over-long integer is rejected as QPACK_DECOMPRESSION_FAILED / decoder-stream error, plus a per-QPACK-stream cumulation-size cap. Consider avoiding the O(N²) full-buffer rescan on each decode().

Reported by zx (Jace) — GitHub: @manus-use