Repository navigation
Add _DuringGC functions for tp_traverse #145921
Description
Feature or enhancement
As seen in https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/13 and capi-workgroup/decisions#95:
There are newly documented restrictions on tp_traverse:
The traversal function must not have any side effects. It must not modify the reference counts of any Python objects nor create or destroy any Python objects.
But, whether a C-API function modifies some reference count is an implementation detail that we can change at any time. Core devs can add logging, an audit event, or a warning, to virtually any function.
This implies that you can't safely use any C API function in tp_traverse, unless it guarantees that it has no side effects.
I propose to add such side-effect-free versions of some functions useful in tp_traverse, with a _DuringGC suffix -- namely:
PyObject_GetTypeData_DuringGCPyObject_GetItemData_DuringGCPyType_GetModuleState_DuringGCPyModule_GetState_DuringGCPyModule_GetToken_DuringGCPyType_GetBaseByToken_DuringGCPyType_GetModule_DuringGCPyType_GetModuleByToken_DuringGC
Compared to their non-suffixed variants, the main difference is that we promise that these will remain safe to use in tp_traverse. The other differences are that these functions cannot fail, and (where applicable) they return borrowed references rather than strong ones.
Also, Py_VISIT, Py_SIZE, PyObject_VisitManagedDict as well as the visitproc that Python passes to tp_traverse will be documented to work like this.
[edit] I'll also include a few ones useful in these: PyObject_TypeCheck, PyType_IsSubtype, PyType_HasFeature, Py{<type>}_Check, Py{<type>}_CheckExact.
Users that call any _DuringGC function (including some type's tp_traverse) must manually must ensure that the type, MRO, and layout of any argument doesn't concurrently change. That means only calling them from tp_traverse.
Activity
[-]Add functions tp_traverse[/-][+]Add _DuringGC functions for tp_traverse[/+]pythongh-145921: Add "_DuringGC" functions for tp_traverse
encukou commentedon Apr 8, 2026
I've merged with a few outstanding issues in
tp_traversedocumentation that Steve pointed out. Both are for pre-existing functionality.On the note “The
tp_traversefunction can be called from any thread.”:And on the traverse function's result:
→
gh-145921: Add "_DuringGC" functions for tp_traverse (GH-145925)
pythongh-145921: Add "_DuringGC" functions for tp_traverse (pythonGH-…
encukou commentedon Oct 22, 2026
@nascheme, maybe you could help with the docs/recommendations here?
nascheme commentedon Oct 23, 2026
In terms of the
tp_traversefunction result, we want the tp_traverse implementations to do whatPy_VISIT()does. I.e. exit early if a non-zero result is returned and return that result. We didn't originally havePy_VISIT(), otherwise maybe we would have just said "you must use Py_VISIT() in tp_traverse".The question of what you can do from tp_traverse and which functions you can call, things have evolved over the years since tp_traverse was introduced. The purpose of tp_traverse is to reveal the
PyObjectpointers that your object directly holds. Originally, this would be struct members of the extended PyObject struct that your type creates. So there originally would not be any CPython API calls needed, you just callPy_VISIT()on the pointers you have in the struct.Because we now have more indirect storage of PyObject pointers, I can see the logic to have
_DuringGCfunctions to access these. My solution would have been to designate a small set of APIs as safe to call within tp_traverse. We cannot start making those APIs sometimes fail (or do "dangerous" things) because it will break the GC. If we wanted new behaviour, we would have to make a new API.pythongh-145921: Clarify return value of tp_traverse