Describe the issue:

This is really just to flag it up in case y'all would be interested to investigate - in case it's a Numpy 2.0 problem. When porting Nipy to Numpy 2.0, I find we are getting multiple segfaults from extensions using compiled C code using the Numpy API. See the reproducer below. Compiling and running with Numpy 1.26.4 gives a clean test run, as has been the case up until now. Is it possible we've run over a reference counting bug in Numpy? Or that, for some reason, Numpy 2.0 has exposed our own reference counting bug, not previously triggered with Numpy < 2?

In the reproducer, the tests nearly always trigger a segfault, with example traceback (from Apple interface) below - but it is not deterministic.

Reproduce the code example:

virtualenv nipy-env
. ./nipy-env/bin/activate
git clone https://github.com/matthew-brett/nipy
cd nipy
git checkout numpy-20-fixes
pip install meson-python ninja cython numpy pytest spin
spin install
pytest nipy/labs/bindings/tests/test_numpy.py

Error message:

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	    0x7ff80def814a __pthread_kill + 10
1   libsystem_pthread.dylib       	    0x7ff80df30ebd pthread_kill + 262
2   libsystem_c.dylib             	    0x7ff80de1c0a8 raise + 24
3   Python                        	       0x109f10002 faulthandler_fatal_error + 405
4   libsystem_platform.dylib      	    0x7ff80df5ffdd _sigtramp + 29
5   ???                           	       0x1095927b0 ???
6   libsystem_c.dylib             	    0x7ff80de56a79 abort + 126
7   libsystem_malloc.dylib        	    0x7ff80dd5b3a9 malloc_vreport + 857
8   libsystem_malloc.dylib        	    0x7ff80dd7ec5b malloc_zone_error + 183
9   libsystem_malloc.dylib        	    0x7ff80dd53da0 tiny_free_list_remove_ptr + 566
10  libsystem_malloc.dylib        	    0x7ff80dd530f3 tiny_free_no_lock + 690
11  libsystem_malloc.dylib        	    0x7ff80dd52d11 free_tiny + 526
12  wrapper.cpython-310-darwin.so 	       0x10f2b670f fffpy_multi_iterator_delete + 31
13  wrapper.cpython-310-darwin.so 	       0x10f2accbc __pyx_pw_4nipy_4labs_8bindings_7wrapper_13pass_vector_via_iterator + 972

Python and NumPy Versions:

2.0.0
3.10.12 (main, Jun 7 2023, 00:45:35) [Clang 14.0.3 (clang-1403.0.22.14.1)]

Runtime Environment:

[{'numpy_version': '2.0.0',
'python': '3.10.12 (main, Jun 7 2023, 00:45:35) [Clang 14.0.3 '
'(clang-1403.0.22.14.1)]',
'uname': uname_result(system='Darwin', node='lockdowns.local', release='23.5.0', version='Darwin Kernel Version 23.5.0: Wed May 1 20:09:52 PDT 2024; root:xnu-10063.121.3~5/RELEASE_X86_64', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2'],
'not_found': ['AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}}]

Context for the issue:

We can't distribute wheels compatible with Numpy 2.0. And I am wondering if this might be a Numpy 2.0 issue that needs to be ironed out before others run into it. (I have no estimate of how likely that is).