Repository navigation
Crash when free-threaded Python is built with --with-trace-refs #124043
Description
Crash report
What happened?
I'm using ThreadPoolExecutor to write tests for free-threading enabled C extensions.
When running concurrent threads larger than CPython crashes when num_workers, CPython crashes.num_futures >= 2 if the test function is complex enough.
The Python is installed via the following configuration with PyDebug enabled:
./configure --prefix="${PWD}/pydev" \
--enable-ipv6 --with-openssl="$(brew --prefix openssl)" \
--with-system-expat --with-system-libmpdec \
--with-assertions --with-pydebug --with-trace-refs \
--disable-gilfrom concurrent.futures import ThreadPoolExecutor
NUM_WORKERS = 32
NUM_FUTURES = 1024
def concurrent_run(func):
with ThreadPoolExecutor(max_workers=NUM_WORKERS) as executor:
for _ in range(NUM_FUTURES):
executor.submit(func)
concurrent_run(lambda : None)Python 3.13.0rc2+ experimental free-threading build (heads/3.13:112b1704fa6, Sep 13 2024, 15:34:48) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from concurrent.futures import ThreadPoolExecutor
>>> NUM_WORKERS = 32
... NUM_FUTURES = 1024
...
... def concurrent_run(func):
... with ThreadPoolExecutor(max_workers=NUM_WORKERS) as executor:
... for _ in range(NUM_FUTURES):
... executor.submit(func)
...
>>> concurrent_run(lambda : None)
Assertion failed: (value == REFCHAIN_VALUE), function _PyRefchain_Remove, file object.c, line 195.
Fatal Python error: Aborted
Thread 0x0000000173e2b000 (most recent call first):
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 304 in __enter__
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 528 in release
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/concurrent/futures/thread.py", line 87 in _worker
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 992 in run
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 1041 in _bootstrap_inner
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 1012 in _bootstrap
Thread 0x0000000172e1f000 (most recent call first):
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/concurrent/futures/thread.py", line 89 in _worker
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 992 in run
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 1041 in _bootstrap_inner
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 1012 in _bootstrap
Thread 0x0000000171e13000 (most recent call first):
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/concurrent/futures/thread.py", line 89 in _worker
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 992 in run
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 1041 in _bootstrap_inner
File "/Users/PanXuehai/Projects/cpython/pydev/lib/python3.13t/threading.py", line 1012 in _bootstrap
Current thread 0x0000000170e07000 (most recent call first):
Assertion failed: (PyCode_Check(f->f_executable)), function _PyFrame_GetCode, file pycore_frame.h, line 81.CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Output from running 'python -VV' on the command line:
Python 3.13.0rc2+ experimental free-threading build (heads/3.13:112b1704fa6, Sep 13 2024, 15:34:48) [Clang 15.0.0 (clang-1500.3.9.4)]
Activity
ZeroIntensity commentedon Sep 13, 2024
At a glance, looks like this could be a data race. I'll take a look.
ZeroIntensity commentedon Sep 13, 2024
Ah,
--with-trace-refsis unsupported under free-threading, so this is a non-issue. As a double check, I confirmed that this doesn't occur with tracerefs disabled. I wonder ifconfigureshould disallow mixing--with-trace-refsand--disable-gil, as this isn't the first case of this happening. See #122921, for example.@encukou, this needs pending.
colesbury commentedon Sep 13, 2024
Let's disable
--with-trace-refsuntil it works so that more people don't run into this[-]Crash for `concurrent.futures.ThreadPoolExecutor` when `num_futures` > `num_workers` on free-threaded Python[/-][+]Crash when building with `--with-trace-refs` in free-threaded Python[/+][-]Crash when building with `--with-trace-refs` in free-threaded Python[/-][+]Crash when free-threaded Python is built with `--with-trace-refs`[/+]ZeroIntensity commentedon Sep 13, 2024
I've created #124078 for this.
gh-124043: Disallow mixing `--with-trace-refs` and `--disable-gil` (#…
pythongh-124043: Disallow mixing `--with-trace-refs` and `--disable-g…
pythongh-124043: Disallow mixing `--with-trace-refs` and `--disable-g…
ZeroIntensity commentedon Sep 23, 2024
Should this get closed, or are we planning on merging the backport before 3.13.0?
encukou commentedon Sep 23, 2024
There's no free-threading before 1.13, so I don't think we should no backports there.
But, do we close issues before the PR is merged? (I assume that it'll be for 3.13.1, next week.)
ZeroIntensity commentedon Sep 23, 2024
I'm not totally sure, I've seen core devs close issues when the only remaining PR is for 3.13, because the branch is locked (but I've also seen other core devs leave issues open, so I don't know).
[3.13] gh-124043: Disallow mixing `--with-trace-refs` and `--disable-…
ZeroIntensity commentedon Sep 24, 2024
I think everything is done here. Thank you for the report, @XuehaiPan!
terryjreedy commentedon Sep 24, 2024
I think issues should be left open until all backports done. The right sidebar of PRs have a
Developmentbutton to make closing the (3.13) PR close the issue. I posted on Discord release-discussions.