Hi Sam, I wonder what's the correct C API calling logic to implement a multi-threading feature in this no_gil CPython. I'm doing some hacking within Modules/gcmodule.c, that I want to mimic gc_get_objects_impl() but for each GC-traced container PyObject, I further call PyObject_GetIter() to obtain all it's inner objects references it holds. I face no problem when executing this logic between tstate = PyGILState_Ensure() and PyGILState_Release(tstate). But apparently it's holding the GIL.
If I don't hold the GIL, the PyObject_GetIter() internally calls _GC_Malloc(), and will seg faults in return mi_heap_calloc(tstate->heaps[mi_heap_tag_gc], nelem, elsize); since the heap structure is messed up.
Then I noticed on PEP 703, about the thread states. In this no_gil CPython 3.9 version, I guess it would be calling _PyThreadState_Swap() to set the thread state ATTACHED, like this:
void *inspect_module_objs(void *arg)
{
PyThreadState *tstate = _PyThreadState_GET();
struct _gilstate_runtime_state *gilstate = &tstate->interp->runtime->gilstate;
if (_PyThreadState_Swap(gilstate, tstate) != NULL)
{
Py_FatalError("non-NULL old thread state");
}
// my threading logic that I don't want to hold GIL
// for loop (...){PyObject_GetIter(each_op) ...}
// ...
PyEval_ReleaseThread(tstate);
}
This inspect_module_objs() is called by PyThread_start_new_thread(inspect_module_objs, args);
However, it seg faults at _PyThreadState_Swap() since the tstate == NULL if you don't call PyGILState_Ensure(). If I hold the GIL before calling _PyThreadState_Swap() it then leads to Py_FatalError("non-NULL old thread state") somehow.
FYI, originally I asked on python forum here before they told be no_gil in 3.13 main stream is not completed, thus I would like to ask here. Thanks you.
Hi Sam, I wonder what's the correct C API calling logic to implement a multi-threading feature in this no_gil CPython. I'm doing some hacking within Modules/gcmodule.c, that I want to mimic
gc_get_objects_impl()but for each GC-traced container PyObject, I further callPyObject_GetIter()to obtain all it's inner objects references it holds. I face no problem when executing this logic betweentstate = PyGILState_Ensure()andPyGILState_Release(tstate). But apparently it's holding the GIL.If I don't hold the GIL, the
PyObject_GetIter()internally calls_GC_Malloc(), and will seg faults inreturn mi_heap_calloc(tstate->heaps[mi_heap_tag_gc], nelem, elsize);since the heap structure is messed up.Then I noticed on PEP 703, about the thread states. In this no_gil CPython 3.9 version, I guess it would be calling
_PyThreadState_Swap()to set the thread stateATTACHED, like this:This
inspect_module_objs()is called byPyThread_start_new_thread(inspect_module_objs, args);However, it seg faults at
_PyThreadState_Swap()since thetstate == NULLif you don't callPyGILState_Ensure(). If I hold the GIL before calling_PyThreadState_Swap()it then leads toPy_FatalError("non-NULL old thread state")somehow.FYI, originally I asked on python forum here before they told be no_gil in 3.13 main stream is not completed, thus I would like to ask here. Thanks you.