Skip to content

Assert on the ALPN extension in test_tunnel_sets_http_11_alpn - #5232

Merged
illia-v merged 5 commits into
urllib3:mainfrom
SEPURI-SAI-KRISHNA:fix-flaky-alpn-assertion
Sep 10, 2026
Merged

illia-v merged 5 commits into
urllib3:mainfrom
SEPURI-SAI-KRISHNA:fix-flaky-alpn-assertion

Conversation

@SEPURI-SAI-KRISHNA

Copy link
Copy Markdown
Contributor

test_tunnel_sets_http_11_alpn asserted against the whole ClientHello:

assert b"http/1.1" in self.buf
assert b"h2" not in self.buf

self.buf is the entire record, most of which is random, so b"h2" turns
up in it by chance. Measuring 3000 handshakes locally (OpenSSL 3.0.13, a
517-byte ClientHello), 7 of them contained b"h2" somewhere, about one in
430. CI is worse: newer OpenSSL sends a post-quantum key share, so the
record is several times larger and the odds scale with it. I hit it twice
in 300 local runs of the test and once on Windows CI.

This parses the ALPN extension out of the ClientHello and compares the
protocol list:

assert client_hello_alpn_protocols(self.buf) == [b"http/1.1"]

That is stricter than what it replaces, it checks http/1.1 is the only
protocol offered, instead of checking its bytes appear somewhere in the
packet, and the failure message becomes readable:

AssertionError: assert [b'h2', b'http/1.1'] == [b'http/1.1']

rather than a hex dump of the ClientHello.

The socket handler now also reads until the record announced by its header
is complete, instead of assuming one packet. A post-quantum ClientHello is
big enough that this is worth not relying on, and an incomplete read would
otherwise just trade one flake for another.

Verification

  • 500 consecutive runs of the test, no failures.
  • Reverting ALPN_PROTOCOLS to ["h2", "http/1.1"] makes it fail, so the
    assertion still catches the regression it is there to catch.
  • The parser returns the right protocol list for ["http/1.1"],
    ["h2", "http/1.1"] and no ALPN at all, over 3000 handshakes.
  • Full test_socketlevel.py: 122 passed, 8 skipped.

The test looked for `http/1.1` and `h2` in the ClientHello as a whole,
but the random bytes, the session id and the key shares can spell either
one out by accident. Locally that made it fail roughly once in 400 runs,
and more often on CI, where the key share carries a post-quantum group
and the record is correspondingly larger.

Parse the ALPN extension and compare the protocols it offers instead.
That is both stable and stricter: it now checks that `http/1.1` is the
only protocol offered, rather than that its bytes appear somewhere. The
handler also reads until the announced record is complete, so a larger
ClientHello split across packets cannot reintroduce the flakiness.
@SEPURI-SAI-KRISHNA

Copy link
Copy Markdown
Contributor Author

The red Ubuntu 3.13 here is unrelated to this change:

MEMORY PROBLEMS test/test_response.py::TestResponse::test_buffer_memory_usage_no_decoding[False-None-read1]
  - Test was limited to 10.5MiB but allocated 11.0MiB

test_response.py runs about 1400 tests before test_socketlevel.py, so nothing here has executed by the time it fails. I opened #5234 for it: the test allows 0.5 MiB of headroom on a 10 MiB body while guarding against a second copy that would take the peak to 20 MiB, which is far tighter than it needs to be.

While setting up Python 3.13 with OpenSSL 3.5.7 to check that, I got a better number for this PR than the one in the description. On that OpenSSL the ClientHello is 1531 bytes rather than 517, and b"h2" turns up in it by chance in 40 of 1500 handshakes, 2.67%, roughly one run in 37, against the one in 430 I measured on OpenSSL 3.0.13. The parser was correct on all 1500. This branch's test also passes there, and test_socketlevel.py as a whole is 122 passed / 8 skipped on 3.13.

@illia-v illia-v left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for looking into this test!

Could we simplify the fix by wrapping the handler socket with an SSL context configured to prefer HTTP/2 over HTTP/1.1, then asserting that ssl.SSLSocket.selected_alpn_protocol returns http/1.1?
That should catch accidentally advertising HTTP/2 without needing to parse the ClientHello.

The handler now completes a TLS handshake with a context that prefers
HTTP/2, so the protocol it negotiates comes back as http/1.1 only when
the client offered nothing else. That is the guarantee the ClientHello
parser was there to give, without the parser.
@SEPURI-SAI-KRISHNA

Copy link
Copy Markdown
Contributor Author

Much better, thanks, done.

The handler now completes a real handshake with a context offering DEFAULT_CERTS["alpn_protocols"], which is ["h2", "http/1.1"], and the test asserts selected_alpn_protocol() == "http/1.1". The server picks the first protocol it prefers that the client also offered, so the assertion fails as soon as the client advertises h2. The ClientHello parser and the struct import are gone, net -38 lines.

Verified with ALPN_PROTOCOLS patched to ["h2", "http/1.1"] to simulate the regression: the test fails with assert 'h2' == 'http/1.1'. 500 consecutive runs clean on 3.12 with OpenSSL 3.0.13 and 500 more on 3.13 with OpenSSL 3.5.7, where the old assertion tripped roughly one run in 37. test_socketlevel.py as a whole is 122 passed, 8 skipped on both.

Comment thread test/with_dummyserver/test_socketlevel.py Outdated
Comment thread test/with_dummyserver/test_socketlevel.py Outdated
@illia-v
illia-v merged commit 5f2a6a8 into urllib3:main Sep 10, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants