Deletion of autoTSSkey during runtime finalization is not safe

@ZeroIntensity

Bug report

The autoTssKey is deleted during _PyRuntimeState_Fini by gilstate_tss_fini. This isn't safe because other threads may try calling PyGILState_Ensure() or PyGILState_GetThisThreadState() concurrently during shutdown.

cpython/Python/pystate.c

Lines 486 to 501 in e9d210b

    
           void 
        
           _PyRuntimeState_Fini(_PyRuntimeState *runtime) 
        
           { 
        
           #ifdef Py_REF_DEBUG 
        
               /* The count is cleared by _Py_FinalizeRefTotal(). */ 
        
               assert(runtime->object_state.interpreter_leaks == 0); 
        
           #endif 
        
               if (gilstate_tss_initialized(runtime)) { 
        
                   gilstate_tss_fini(runtime); 
        
               } 
        
               if (PyThread_tss_is_created(&runtime->trashTSSkey)) { 
        
                   PyThread_tss_delete(&runtime->trashTSSkey); 
        
               } 
        
           }

We can:

Convert autoTssKey to a _Py_thread_local like _Py_tss_tstate, which doesn't require deletion
Don't delete autoTssKey at runtime finalization

My preference is for the first option.

cc @ZeroIntensity @ericsnowcurrently @gpshead

Linked PRs

gh-131185: Use a proper thread-local for cached thread states #132510

Making it a thread-local sounds reasonable, but I think we also need to deal with NULL auto interpreter states in order to get PyGILState_Ensure fully working (as in, hanging the thread) post-finalization.

My other concern (which might be an issue regardless of autoTSSkey) is what happens to the thread state when the thread is hung. Does it just get leaked?

...what happens to the thread state when the thread is hung. Does it just get leaked?

No, the thread state gets deleted:

cpython/Python/pystate.c

Line 3057 in ca7fd81

decref_threadstate(tstate_impl);

But I'm not really sure it matters because:

The process is about to exit, so anything you "leak" is going to disappear soon
The native thread will still be leaked
Any Python objects referenced by the thread's stack will be leaked. This is true regardless of whether we kill the thread via pthread_exit() or block it via PyThread_hang_thread().

The process is about to exit, so anything you "leak" is going to disappear soon

I'm thinking about embedders that don't necessarily exit after Py_Finalize, but yeah, I don't think it matters much either. I would assume that most embedders keep the runtime alive for the duration of their app as well.

gh-131185: Use a proper thread-local for cached thread states (gh-132510

The broader issue of unprotected runtime state at finalization is handled in gh-134307.

@ZeroIntensity

Thanks for tackling this, @ZeroIntensity!

pythongh-131185: Use a proper thread-local for cached thread states (p…

Navigation Menu

Deletion of autoTSSkey during runtime finalization is not safe #131185

Bug report

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	void
	_PyRuntimeState_Fini(_PyRuntimeState *runtime)
	{
	#ifdef Py_REF_DEBUG
	/* The count is cleared by _Py_FinalizeRefTotal(). */
	assert(runtime->object_state.interpreter_leaks == 0);
	#endif

	if (gilstate_tss_initialized(runtime)) {
	gilstate_tss_fini(runtime);
	}

	if (PyThread_tss_is_created(&runtime->trashTSSkey)) {
	PyThread_tss_delete(&runtime->trashTSSkey);
	}
	}

Deletion of autoTSSkey during runtime finalization is not safe #131185

Description

Bug report

Linked PRs

Activity

ZeroIntensity commented on Mar 13, 2025

colesbury commented on Mar 14, 2025

ZeroIntensity commented on Mar 14, 2025

ericsnowcurrently commented on May 21, 2025

ericsnowcurrently commented on May 21, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions