Add Python 3.14 support#106
Open
jjviscomi wants to merge 1 commit into
Open
Conversation
Python 3.14 reshuffles the bytecode in three ways that touch Atheris's
instrumentation logic:
* New non-popping conditional jumps `JUMP_IF_TRUE` / `JUMP_IF_FALSE`,
relative-forward.
* `LOAD_SMALL_INT` is now used to push small-integer literals; previously
every literal flowed through `LOAD_CONST`.
* Several existing opcodes were renamed or removed (e.g. `RETURN_CONST`,
`LOAD_METHOD`, `BUILD_CONST_KEY_MAP`); none of which Atheris referenced
by name, so no change is required there.
Source changes:
* Allow `PYTHON_VERSION == (3, 14)` past the version gate; update the
error message and the supported-versions docstring.
* Categorize the two new conditional jumps under `CONDITIONAL_JUMPS` and
`HAVE_REL_REFERENCE` so offset rewriting handles them.
* Introduce `CONST_LOADS` and use it in the constant-compare
instrumentation path so `LOAD_SMALL_INT` is recognized as a constant
push alongside `LOAD_CONST`.
Test change to `test_extended_arg` (minimal scope):
* Replace `for inst in original_code.co_code: assertNotEqual(dis.opname[inst], "EXTENDED_ARG")`
with iteration over `dis.get_instructions(code)`. The previous loop
indexed `dis.opname` with raw `co_code` bytes, which include argument
bytes; that worked on Python 3.11-3.13 by luck (no argument byte
coincided with the EXTENDED_ARG opcode value) but produces a false
positive on Python 3.14, where a `COMPARE_OP` argument byte equals
the new EXTENDED_ARG opcode number (69). `dis.get_instructions` is
the documented correct API for decoding bytecode.
* Drop one `pass` from the fixture (253 to 252). The function was
calibrated so the original bytecode stayed below the EXTENDED_ARG
threshold (~256-byte jumps); Python 3.14's slightly more verbose
bytecode pushed 253 passes past that threshold, which would defeat
the test's "original lacks EXTENDED_ARG; patching inserts one"
invariant. At 252 passes the invariant holds identically on both
Python 3.13 and 3.14 (verified empirically). The test's intent and
both assertions are otherwise unchanged.
Infrastructure:
* README, CI matrix, and the manylinux Dockerfile now include 3.14.
Validated against the full Atheris test suite on both Python 3.13.12 and
3.14.2: 149 / 149 tests pass on each. The lone pyinstaller_coverage_test
failure on both versions is unrelated to 3.14, it requires PyInstaller
to be installed (handled by `run_tests.sh` in CI).
Closes google#105
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Python 3.14 support to Atheris. Closes the "Failed to build" symptom
reported in #105 and brings the supported version range to 3.11-3.14.
What changed in Python 3.14 that mattered
Three bytecode-level changes touch Atheris's instrumentation:
JUMP_IF_TRUE/JUMP_IF_FALSE(relative-forward)LOAD_SMALL_INTis now the loader for small-integer literalsLOAD_CONSTand missed small-int literalsRETURN_CONST,LOAD_METHOD,BUILD_CONST_KEY_MAP, etc.)Source changes
src/version_dependent.py: allow(3, 14)past the version gate, categorize the new jumps underCONDITIONAL_JUMPS+HAVE_REL_REFERENCE, introduce aCONST_LOADSlist (LOAD_CONSTon every version, plusLOAD_SMALL_INTon 3.14+).src/instrument_bytecode.py: useCONST_LOADSinstead of the hard-coded"LOAD_CONST"string in the constant-compare detection.README.md,.github/workflows/builds.yaml,deployment/Dockerfile: bump to 3.11-3.14.Test change to
test_extended_arg(justification)Changing an established test deserves explicit justification. Here is what I found and why this is the minimum-scope correction.
The two problems with the test on Python 3.14
Problem 1: pre-existing iteration bug. The original loop:
iterates
co_codebyte-by-byte and indexesdis.opnamewith each byte.co_codeis[opcode, arg, opcode, arg, ...], so half the bytes are arguments, not opcodes. The loop was producing the correct result on Python 3.11-3.13 by luck (no argument byte happened to equal the EXTENDED_ARG opcode value).On Python 3.14, EXTENDED_ARG is opcode 69, and a
COMPARE_OPargument byte in this fixture's bytecode also equals 69. The loop reports a false positive.dis.get_instructions(code)is the documented correct API for decoding bytecode (see thedismodule docs); switching to it makes the iteration semantically correct without changing what the test asserts.Problem 2: fixture size calibrated to the EXTENDED_ARG threshold. The fixture's 253
passstatements were calibrated so the original bytecode stayed just below the 255-byte jump threshold (~256 bytes of co_code). Python 3.14's slightly more verbose bytecode (new opcodes, different inline-cache layout) pushes 253 passes past that threshold; the original now genuinely contains an EXTENDED_ARG, defeating the test's invariant.I swept the threshold empirically:
252 is the unique count where the test's "original lacks EXTENDED_ARG; patching adds one" invariant holds identically on both Python 3.13 and 3.14.
The fix
Two small, intent-preserving changes to the test:
dis.get_instructions(code)instead of rawco_codebytes, in both assertion loops. The assertion targets and structure are unchanged.passstatements.The test's docstring ("Tests that we can handle the insertion of new EXTENDED_ARG instructions"), assertion semantics, and both checks are preserved unchanged. The bytecode invariant the test depends on is verified empirically on both supported newest Pythons.
Acknowledged residual brittleness
The fixture size remains tuned to a specific bytecode-layout window. Future Python releases will eventually shift the threshold again. A self-calibrating fixture (compute the EXTENDED_ARG threshold at runtime, generate the function size accordingly) would be more robust. That refactor is out of scope for adding 3.14 support and would be a separate, focused PR.
Verification
Ran the full Atheris test suite on both Python 3.13.12 (regression) and 3.14.2 (new support):
The
pyinstaller_coverage_testfailure is identical on both versions and unrelated to Python 3.14: it requires PyInstaller in the venv, whichrun_tests.shinstalls in CI but my local one-off invocation did not.Out of scope
sys.monitoring(PEP 669, available since 3.12). That is the architectural fix that would eliminate per-Python-version bytecode work and is a substantial separate effort.test_extended_argfixture to be robust against future bytecode-layout changes (see "residual brittleness" above).Closes
Closes #105.