Skip to content

PyObject_Realloc(NULL, size) crashes free-threaded 32-bit Python 3.15.0b2 #151297

Closed
Closed
PyObject_Realloc(NULL, size) crashes free-threaded 32-bit Python 3.15.0b2#151297
Labels
topic-ctypestopic-free-threadingtype-crashA hard crash of the interpreter, possibly with a core dump

Description

@godlygeek
godlygeek
opened on Jun 11, 2026

Crash report

What happened?

I haven't managed to reproduce this with a build of CPython from source, but it reproduces using the manylinux images shipped by the PyPA on an x86-64 machine:

docker run -it quay.io/pypa/manylinux_2_34_i686 /usr/local/bin/python3.15t -c 'import ctypes; PyObject_Realloc = ctypes.pythonapi.PyObject_Realloc; PyObject_Realloc.argtypes = [ctypes.c_void_p, ctypes.c_size_t]; PyObject_Realloc.restype = ctypes.c_void_p; print(PyObject_Realloc(None, 984))'

It only crashes for python3.15t, not for python3.15 or python3.14t.

It only crashes for the i686 image, not the x86-64 image.

(gdb) bt
#0  0x5ebd5a4e in ?? ()
#1  0x5ebd7a21 in PyObject_Realloc ()
#2  0xf2cc37dc in ffi_call_i386 () at ../src/x86/sysv.S:121
...

(gdb) x/30i 0x5ebd5a4e - 50
   0x5ebd5a1c:  nopl   0x0(%eax)
   0x5ebd5a20:  mov    0xc(%ebp),%edi
   0x5ebd5a23:  mov    (%edi,%edx,1),%edi
   0x5ebd5a26:  mov    %edi,(%esi,%edx,1)
   0x5ebd5a29:  mov    0xc(%ebp),%edi
   0x5ebd5a2c:  add    $0x4,%edx
   0x5ebd5a2f:  cmp    %edx,%eax
   0x5ebd5a31:  jne    0x5ebd5a20
   0x5ebd5a33:  lea    (%esi,%eax,1),%edx
   0x5ebd5a36:  add    %edi,%eax
   0x5ebd5a38:  sub    $0x4,%esp
   0x5ebd5a3b:  push   %ecx
   0x5ebd5a3c:  push   %eax
   0x5ebd5a3d:  push   %edx
   0x5ebd5a3e:  call   0x5edd6220 <memcpy@plt>
   0x5ebd5a43:  add    $0x10,%esp
   0x5ebd5a46:  lea    -0x1(%edi),%eax
   0x5ebd5a49:  and    $0xffc00000,%eax
=> 0x5ebd5a4e:  mov    0x6c(%eax),%edx
   0x5ebd5a51:  sub    %eax,%edi
   0x5ebd5a53:  shr    $0x9,%edi
   0x5ebd5a56:  and    $0xffffffc0,%edi
   0x5ebd5a59:  lea    (%eax,%edi,1),%ecx
   0x5ebd5a5c:  sub    0x74(%eax,%edi,1),%ecx
   0x5ebd5a60:  mov    %gs:0x0,%edi
   0x5ebd5a67:  add    $0x70,%ecx
   0x5ebd5a6a:  cmp    %edx,%edi
   0x5ebd5a6c:  jne    0x5ebd5acf
   0x5ebd5a6e:  cmpb   $0x0,0xe(%ecx)
   0x5ebd5a72:  jne    0x5ebd5ad6

It looks like it's crashing after the memcpy call that realloc makes to copy from the old buffer to the new. The only other thing _PyObject_MiRealloc does after that point is call mi_free(ptr) to free the old pointer, but that should be special casing null...

The crash is from trying to access:

(gdb) p $_siginfo._sifields._sigfault.si_addr
$1 = (void *) 0xffc0006c

The address's high byte being 0xFF on a 32-bit address space makes me think something has wrapped around by subtracting from the null pointer to find the malloc housekeeping structures or something, but I haven't figured out why, since all of the code does seem to be properly guarded against null pointers.

CPython versions tested on:

3.15

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.15.0b2 free-threading build (main, Jun 6 2026, 08:05:01) [Clang 22.1.7 ]

Linked PRs

Activity

  1. godlygeek
    added
    type-crashA hard crash of the interpreter, possibly with a core dump
    on Jun 11, 2026
  2. godlygeek
    changed the title [-]Crash in free-threaded 32-bit Python 3.15.0b2 calling PyObject_Realloc(NULL, size)[/-] [+]PyObject_Realloc(NULL, size) crashes free-threaded 32-bit Python 3.15.0b2[/+] on Jun 11, 2026

  3. godlygeek commented on Jun 11, 2026

    @godlygeek
    godlygeek
    on Jun 11, 2026
    ContributorAuthor

    Ooh, it also seems to be intermittent:

    [root@bacf47817bc8 /]# /usr/local/bin/python3.15t -c 'import ctypes; PyObject_Realloc = ctypes.pythonapi.PyObject_Realloc; PyObject_Realloc.argtypes = [ctypes.c_void_p, ctypes.c_size_t]; PyObject_Realloc.restype = ctypes.c_void_p; print(PyObject_Realloc(None, 984))'
Segmentation fault (core dumped)
[root@bacf47817bc8 /]# /usr/local/bin/python3.15t -c 'import ctypes; PyObject_Realloc = ctypes.pythonapi.PyObject_Realloc; PyObject_Realloc.argtypes = [ctypes.c_void_p, ctypes.c_size_t]; PyObject_Realloc.restype = ctypes.c_void_p; print(PyObject_Realloc(None, 984))'
3777305600
[root@bacf47817bc8 /]# /usr/local/bin/python3.15t -c 'import ctypes; PyObject_Realloc = ctypes.pythonapi.PyObject_Realloc; PyObject_Realloc.argtypes = [ctypes.c_void_p, ctypes.c_size_t]; PyObject_Realloc.restype = ctypes.c_void_p; print(PyObject_Realloc(None, 984))'
3911523328
[root@bacf47817bc8 /]# /usr/local/bin/python3.15t -c 'import ctypes; PyObject_Realloc = ctypes.pythonapi.PyObject_Realloc; PyObject_Realloc.argtypes = [ctypes.c_void_p, ctypes.c_size_t]; PyObject_Realloc.restype = ctypes.c_void_p; print(PyObject_Realloc(None, 984))'
Segmentation fault (core dumped)
  4. sergey-miryanov
    added
    topic-ctypes
    topic-free-threading
    on Jun 11, 2026

  5. sergey-miryanov commented on Jun 11, 2026

    @sergey-miryanov
    sergey-miryanov
    on Jun 11, 2026
    Contributor

    Using ctypes to achieve non-standard behavior is a bit too hacky and falls outside our project's scope.

    Furthermore, Py_None is a statically allocated object, so calling PyObject_Realloc on it results in undefined behavior.

    Given this, I'm not sure we should try to do anything about it here.

  6. sergey-miryanov
    added
    pendingThe issue will be closed if no feedback is provided
    on Jun 11, 2026

  7. godlygeek commented on Jun 11, 2026

    @godlygeek
    godlygeek
    on Jun 11, 2026
    ContributorAuthor

    ctypes isn't relevant here, you can reproduce this with a C extension if you prefer. I just figured a reproducer that doesn't require compiling extra code would be easier to work with.

    This does not try to reallocate Py_None. When a ctypes function that expects a pointer is passed None, the FFI layer passes NULL down to the called function.

  8. godlygeek commented on Jun 11, 2026

    @godlygeek
    godlygeek
    on Jun 11, 2026
    ContributorAuthor

    I've just figured out how to reproduce the crash with a debug build: it happens with Clang but not with GCC!

    It's possible this is a compiler bug rather than a CPython bug...

    mi_free (p=0x0) at Objects/mimalloc/alloc.c:576
576       const bool          is_local= (_mi_prim_thread_id() == mi_atomic_load_relaxed(&segment->thread_id));
(gdb) bt
#0  mi_free (p=0x0) at Objects/mimalloc/alloc.c:576
#1  _PyObject_MiRealloc (ctx=<optimized out>, ptr=0x0, nbytes=984) at Objects/obmalloc.c:368
#2  0x08135e40 in PyObject_Realloc (ptr=0x0, new_size=984) at Objects/obmalloc.c:1728
#3  0xf5c187dc in ffi_call_i386 () at ../src/x86/sysv.S:121

    mi_free (p=0x0) at Objects/mimalloc/alloc.c:576
576       const bool          is_local= (_mi_prim_thread_id() == mi_atomic_load_relaxed(&segment->thread_id));

(gdb) list
571     // fast path written carefully to prevent spilling on the stack
572     void mi_free(void* p) mi_attr_noexcept
573     {
574       if mi_unlikely(p == NULL) return;
575       mi_segment_t* const segment = mi_checked_ptr_segment(p,"mi_free");
576       const bool          is_local= (_mi_prim_thread_id() == mi_atomic_load_relaxed(&segment->thread_id));
577       mi_page_t* const    page    = _mi_segment_page_of(segment, p);
578
579       if mi_likely(is_local) {                       // thread-local free?
580         if mi_likely(page->flags.full_aligned == 0)  // and it is not a full page (full pages need to move from the full bin), nor has aligned blocks (aligned blocks need to be unaligned)
(gdb) p p
$1 = (void *) 0x0

    Somehow, despite p being NULL, the Clang-compiled binary has wound up on line 576 of mi_free, despite that function being gated with if mi_unlikely(p == NULL) return;

    Either this is a compiler bug, or somewhere we performed an operation on p that would be UB if performed on a NULL pointer and so the compiler has branched assuming that p must not be NULL...

  9. sergey-miryanov commented on Jun 11, 2026

    @sergey-miryanov
    sergey-miryanov
    on Jun 11, 2026
    Contributor

    Thanks for clarification.

  10. ZeroIntensity
    removed
    pendingThe issue will be closed if no feedback is provided
    on Jun 11, 2026

  11. ZeroIntensity commented on Jun 11, 2026

    @ZeroIntensity
    ZeroIntensity
    on Jun 11, 2026
    Member

    Yeah, this feels a lot like a compiler bug. That mi_unlikely might be doing something suspicious when aggressively compiled by Clang. I would try removing that and seeing if the crash magically disappears.

    I've just figured out how to reproduce the crash with a debug build: it happens with Clang but not with GCC!

    Is this still i686? I've been trying to reproduce this on x86-64, but I haven't been able to.

  12. godlygeek commented on Jun 11, 2026

    @godlygeek
    godlygeek
    on Jun 11, 2026
    ContributorAuthor

    Yes, still needs i686. I've been reproducing it in a quay.io/pypa/manylinux_2_34_i686 container, run from a Linux x86-64 machine.

  13. godlygeek commented on Jun 11, 2026

    @godlygeek
    godlygeek
    on Jun 11, 2026
    ContributorAuthor

    Specifically I got it reproducing with:

    docker run -v $PWD:/workspace -it quay.io/pypa/manylinux_2_34_i686

    and then

    yum install clang
cd /tmp
git clone https://github.com/python/cpython
cd cpython
CC=clang CFLAGS_NODIST='-g -O2 -D_FORTIFY_SOURCE=2' ./configure --disable-shared --disable-gil
make -j
gdb --args ./python -c 'import ctypes; PyObject_Realloc = ctypes.pythonapi.PyObject_Realloc; PyObject_Realloc.argtypes = [ctypes.c_void_p, ctypes.c_size_t]; PyObject_Realloc.restype = ctypes.c_void_p; print(PyObject_Realloc(None, 984))'

  14. godlygeek commented on Jun 11, 2026

    @godlygeek
    godlygeek
    on Jun 11, 2026
    ContributorAuthor

    Ah, and here's our root cause: /usr/include/string.h does:

    /* Copy N bytes of SRC to DEST.  */
extern void *memcpy (void *__restrict __dest, const void *__restrict __src,
		     size_t __n) __THROW __nonnull ((1, 2));

    That declares that both __dest and __src must be non-null.

    So, when _PyObject_MiRealloc does _mi_memcpy(newp, ptr, 0); the compiler decides it's now free to assume that ptr is non-null, so when it inlines in mi_free(ptr) it drops the if mi_unlikely(p == NULL) return; check as something that can never be true.

  15. ZeroIntensity commented on Jun 11, 2026

    @ZeroIntensity
    ZeroIntensity
    on Jun 11, 2026
    Member

    Makes sense, but I am still very confused as to why this doesn't reproduce on x86-64.

  16. ZeroIntensity
    added 2 commits that reference this issue on Jun 12, 2026

    gh-151297: Fix undefined behavior in `_PyObject_MiRealloc` (GH-151358)

    c375992

    [3.15] gh-151297: Fix undefined behavior in `_PyObject_MiRealloc` (GH…

    a2e5516
  17. ZeroIntensity
    closed this as completedon Jun 12, 2026
  18. philthompson10
    added a commit that references this issue on Jun 17, 2026

    pythongh-151297: Fix undefined behavior in `_PyObject_MiRealloc` (pyt…

    7adb0ef
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic-ctypestopic-free-threadingtype-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions