Repository navigation
C API: PyGC_Disable() not respected #116604
Description
Bug report
Bug description:
The Python C API provides the function PyGC_Disable() to temporarily disable garbage collection. Calling it causes PyGC_Collect() to become a no-op. So far so good.
Unfortunately, CPython >= 3.12 no longer respects this flag beyond direct calls to PyGC_Collect(). Let's consider, for example, the operation PyErr_CheckSignals() at line 1773. This calls _Py_RunGC, which ignores the GC enabled flag. And this indeed happens! In fact, I just spent quite a bit of time debugging an issue where the GC was supposed to be disabled for a brief moment, and yet it runs on Python 3.12. (Whether disabling the GC for a brief period of time is a good design pattern is another discussion. I would like to steer the discussion away from this and focus on documented API behavior.)
The PyErr_CheckSignals() function is called from 135 locations in the CPython codebase including very common ones like PyObject_Str(), so making any fixes in callers of this API does not look feasible.
The flag-ignoring _Py_RunGC() function is only called by two places: besides the mentioned PyErr_CheckSignals(), there is also _Py_HandlePending() in Python/ceval_gil.c.
To restore the documented behavior, I see three options:
_Py_RunGC()could be modified to exit immediately if theenabledflag is set to zero.- The implementation of
_Py_HandlePending()andPyErr_CheckSignals()could be modified to checkPyGC_IsEnabled()before calling_Py_RunGC(). - Something is setting
_PY_GC_SCHEDULED_BIT, and that causes the implementation to enter_Py_RunGC(). I'm not really sure about how that works, but perhaps this flag could be cleared inPyGC_Disable(), and Python could then avoid setting the flag.
(Tagging @pablogsal and @swtaarrs, who commited changes to the relevant code.)
CPython versions tested on:
3.12
Operating systems tested on:
Linux
Activity
pablogsal commentedon Mar 11, 2024
@wjakob can you confirm if this affects 3.12?
wjakob commentedon Mar 11, 2024
@pablogsal Confirmed. I am on 3.12.2.
PS: I realized that my original bug description had a
> 3.12instead of>= 3.12. Fixed now.pablogsal commentedon Mar 11, 2024
I will investigate this shortly but if it affects 3.12 I understand exactly what is happening 👍
pythongh-116604: Correctly honor the gc status when calling _Py_RunGC
pythongh-116604: Correctly honor the gc status when calling _Py_RunGC
pythongh-116604: Correctly honor the gc status when calling _Py_RunGC
pythongh-116604: Correctly honor the gc status when calling _Py_RunGC
gh-116604: Correctly honor the gc status when calling _Py_RunGC (#116628
[3.12] pythongh-116604: Correctly honor the gc status when calling _P…
pythongh-116604: Fix test_gc on free-threaded build
pythongh-116604: Fix test_gc on free-threaded build
pythongh-116604: Check for `gcstate->enabled` in _Py_RunGC in free-th…
gh-116604: Check for `gcstate->enabled` in _Py_RunGC in free-threaded…
Merge branch 'pythongh-116604-gc-free-threaded' into nogil-integration
[3.12] pythongh-116604: Correctly honor the gc status when calling _P…
gh-116604: Fix test_gc on free-threaded build (#116662)
pythongh-116604: Fix test_gc on free-threaded build (python#116662)
pythongh-116604: Correctly honor the gc status when calling _Py_RunGC (…
pythongh-116604: Check for `gcstate->enabled` in _Py_RunGC in free-th…
pythongh-116604: Fix test_gc on free-threaded build (python#116662)
pythongh-116604: Correctly honor the gc status when calling _Py_RunGC (…
pythongh-116604: Check for `gcstate->enabled` in _Py_RunGC in free-th…
pythongh-116604: Fix test_gc on free-threaded build (python#116662)
wjakob commentedon Jun 11, 2024
@pablogsal Should this issue be closed?
pablogsal commentedon Jun 11, 2024
Yep