Add NumPy 2.0 support #2599

aganders3 · 2024-06-07T20:06:22Z

This introduces numpy 2.0 support. napari has started looking into support, and ran into a few small issues using vispy testing against it. See the napari zulip thread for some discussion and relevant links.

The process for adding support in this PR was:

Run the ruff numpy 2.0 support check and fix:

ruff check vispy --select NPY201
ruff check vispy --select NPY201 --fix

Run tests, fix issues and warnings
Update for changes to behavior of the copy kwarg in np.array
- Add vispy.util.np_copy_if_needed value (False for numpy < 1.28, None for numpy >= 2.0)
- Search for copy=False used in no.array and remove it
- Search for copy=copy used in np.array, replace with copy=vispy.util.np_copy_if_needed
- See references for more information
Update build configuration for Cython code
- Previously, it was important to link against the oldest compatible numpy version to ensure
  forward compatibility. This was accomplished by depending on
  oldest-supported-numpy.
- Now, you can just depend on numpy>=2.0.0rc1, and the build will be compatible with both 1.x
  and 2.x.
- This is updated in the [build-system] section of pyproject.toml
- See Depending on NumPy for more information

I think this is ready for review, but I'm going to try creating a wheel and using to run napari tests with numpy 2.0 as well. I'll mark it non-draft after I do that.

aganders3 · 2024-06-07T20:12:38Z

Ah, wheel builds are failing, which I did not test. It looks like maybe the NumPy 2.0-compatible wheels will need to be Python 3.9+.

djhoese · 2024-06-07T20:16:50Z

Do we build 3.8? I'm ok only doing 3.9+

aganders3 · 2024-06-08T01:12:10Z

Yes, as of 0.14.2 there are 3.8 wheels. The first thing I noticed though was the sdist was apparently being built with 3.7 😮.

aganders3 · 2024-06-10T14:12:24Z

I'm marking this ready for review, as I was able to run napari (+tests) without noticing any (vispy-related) issues.

It would be great to have this in before the numpy 2.0 release on June 16, but of course I want to make sure it's actually ready so any review is appreciated!

aganders3 · 2024-06-10T16:10:27Z

Running tests in napari does reveal some deprecation warnings coming through vispy:

======================================================================================================== warnings summary ========================================================================================================
napari/layers/shapes/_tests/test_shapes_mouse_bindings.py: 49 warnings
  /Users/aandersoniii/src/napari/numpy2-venv/lib/python3.11/site-packages/vispy/geometry/triangulation.py:679: DeprecationWarning: Arrays of 2-dimensional vectors are deprecated. Use arrays of 3-dimensional vectors instead. (deprecated in NumPy 2.0)
    return np.cross(B-A, C-B) > 0

napari/layers/shapes/_tests/test_shapes_mouse_bindings.py: 18 warnings
  /Users/aandersoniii/src/napari/numpy2-venv/lib/python3.11/site-packages/vispy/geometry/triangulation.py:742: DeprecationWarning: Arrays of 2-dimensional vectors are deprecated. Use arrays of 3-dimensional vectors instead. (deprecated in NumPy 2.0)
    c = np.cross(v1, v2)  # positive if v1 is CW from v2

There is some discussion here about what should be used instead:
numpy/numpy#26620

brisvag · 2024-06-12T08:47:02Z

Looks like we should wait for cross2d to be exposed; let's check back in a bit!

aganders3 · 2024-06-12T10:40:51Z

I'm not sure we'd want to use it, as that would make it 2.0-only, right?

brisvag · 2024-06-12T11:46:30Z

True... I guess we should use their example like you did in the napari PR:

cross2d = arr1[..., 0] * arr2[..., 1] - arr1[..., 1] * arr2[..., 0]

A bit ugly to lose the explicit "cross", but oh well

aganders3 · 2024-06-12T12:00:58Z

Yeah, it's unfortunate but I understand why it's changing. My concern is if any of these functions can take 2D or 3D input, but the behavior is quite different so I doubt it.

I guess we could make our own cross2d to drop in.

jni · 2024-06-13T06:57:04Z

I guess we could make our own cross2d to drop in.

👆 this is the right way I think. Abstract away the unreadability and make clear the intent of the code.

brisvag

Great, I like the new function, much more readable.

brisvag · 2024-06-14T14:55:25Z

pyproject.toml

+    # see https://numpy.org/devdocs/dev/depending_on_numpy.html#numpy-2-0-specific-advice
+    "numpy>=2.0.0rc2",


In the link they suggest adding a test against the oldest supported numpy version. Am I understanding correctly that before we were getting this "for free" because we compiled with the oldest numpy, but now we don't anymore and so maybe we should add this test?

I don't think we were getting that for free before either, as I believe this should only affect the build configuration. It seems like they're recommending new tests added to test against the minimum supported version of numpy. Do you think we should add that here?

What's happening in this branch now is that we're building against numpy 2.0.0rc2 and testing against the latest released version (1.26.4).

Got it... No need to add them here then I guess :) We can do in a follow up!

I think we'll need to do some testing to figure out what the minimum supported version of numpy should even be. Right now it's left unspecified in setup.py.

brisvag · 2024-06-14T15:03:22Z

vispy/visuals/text/_sdf_cpu.pyx

 from libc.math cimport sqrt
 cimport cython

+np.import_array()


This is also from ruff fixes?

Actually no, this is from a warning. I thought it was related to numpy 2.0, but now that I look maybe not.

See this example in the Cython docs:

# "cimport" is used to import special compile-time information # about the numpy module (this is stored in a file numpy.pxd which is # distributed with Numpy). # Here we've used the name "cnp" to make it easier to understand what # comes from the cimported module and what comes from the imported module, # however you can use the same name for both if you wish. cimport numpy as cnp # It's necessary to call "import_array" if you use any part of the # numpy PyArray_* API. From Cython 3, accessing attributes like # ".shape" on a typed Numpy array use this API. Therefore we recommend # always calling "import_array" whenever you "cimport numpy" cnp.import_array()

jni · 2024-06-15T09:16:34Z

Since tests are passing, we should probably merge and release this sooner rather than later since NumPy 2 is going to be out in a couple of days... 💣 😅

brisvag · 2024-06-17T09:53:52Z

Yep, seems good to me. Let's go!

brisvag · 2024-06-17T12:44:52Z

Released 0.14.3 so we can join the big "update to support numpy 2 party" that all projects are currently participating it :P

aganders3 · 2024-06-17T13:10:41Z

Thanks! Fingers are crossed 😬

Alternative to: #2549 This PR drops the monkeypatch because: - The fix in ctypes for cdlls in bigsur is out in python >3.8: https://bugs.python.org/issue41100 - The remaining issue was python 3.8, which has been dropped in #2599 Also, changes the name of the library to the proper `Quartz`, as per: https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/OSX_Technology_Overview/SystemFrameworks/SystemFrameworks.html The lower case may work due with case insensitive file system, but as noted by Ashley, with caching, it may fail.

aganders3 added 5 commits June 6, 2024 12:17

Initial numpy 2.0 support

07a8393

Simplify numpy copy-if-needed logic, revert verbose testing flag

173091d

Improve formatting on fixed row_stack->vstack

7e4afb0

Use short-circuit syntax for np_copy_if_needed

de8b197

Remove a few more copy=False from np.array calls

4c1eada

Try to bump minimum version to 3.9

ed66cc7

aganders3 marked this pull request as ready for review June 10, 2024 14:11

andy-sweet mentioned this pull request Jun 11, 2024

More numpy 2.0 related fixes napari/napari#6974

Merged

Add _cross_2d to replace deprecated 2D behavior of np.cross

25bde91

aganders3 force-pushed the numpy-2.0 branch from 842d785 to 25bde91 Compare June 13, 2024 15:49

aganders3 mentioned this pull request Jun 14, 2024

Fix MacOs 14 Sonoma dlopen cache lookup #2549

Closed

brisvag approved these changes Jun 14, 2024

View reviewed changes

brisvag merged commit 44dad2f into vispy:main Jun 17, 2024

aganders3 deleted the numpy-2.0 branch June 17, 2024 13:10

rgommers mentioned this pull request Jun 17, 2024

Ecosystem compatibility with numpy 2.0 numpy/numpy#26191

Closed

psobolewskiPhD mentioned this pull request Jan 21, 2025

Ensure proper quartz cdll loading on macOS (cocoapy) #2635

Merged

		# see https://numpy.org/devdocs/dev/depending_on_numpy.html#numpy-2-0-specific-advice
		"numpy>=2.0.0rc2",

Add NumPy 2.0 support #2599

Add NumPy 2.0 support #2599

Uh oh!

Conversation

aganders3 commented Jun 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aganders3 commented Jun 7, 2024

Uh oh!

djhoese commented Jun 7, 2024

Uh oh!

aganders3 commented Jun 8, 2024

Uh oh!

aganders3 commented Jun 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aganders3 commented Jun 10, 2024

Uh oh!

brisvag commented Jun 12, 2024

Uh oh!

aganders3 commented Jun 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brisvag commented Jun 12, 2024

Uh oh!

aganders3 commented Jun 12, 2024

Uh oh!

jni commented Jun 13, 2024

Uh oh!

brisvag left a comment

Choose a reason for hiding this comment

Uh oh!

brisvag Jun 14, 2024

Choose a reason for hiding this comment

Uh oh!

aganders3 Jun 14, 2024

Choose a reason for hiding this comment

Uh oh!

aganders3 Jun 14, 2024

Choose a reason for hiding this comment

Uh oh!

brisvag Jun 14, 2024

Choose a reason for hiding this comment

Uh oh!

aganders3 Jun 14, 2024

Choose a reason for hiding this comment

Uh oh!

brisvag Jun 14, 2024

Choose a reason for hiding this comment

Uh oh!

aganders3 Jun 14, 2024

Choose a reason for hiding this comment

Uh oh!

jni commented Jun 15, 2024

Uh oh!

brisvag commented Jun 17, 2024

Uh oh!

brisvag commented Jun 17, 2024

Uh oh!

aganders3 commented Jun 17, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

aganders3 commented Jun 7, 2024 •

edited

Loading

aganders3 commented Jun 10, 2024 •

edited

Loading

aganders3 commented Jun 12, 2024 •

edited

Loading