Numpy 2 compatibility and the numpy C API #3024
Description
Sorry if you'd rather have this kind of thing on groups.io. I wasn't really sure where this fell on the line between these reporting spaces. Anyway...
I think numpy 2 (not yet released) was recently updated to require that usage of the C API include calling np.import_array() in a Cython extension. I'm working on updating my unstable CI for one of my packages (geoxarray) to pass with numpy 2.0 and this is the current thing I'm stuck on when rasterio is imported. Github search isn't showing me any usage of numpy.import_array or similar in rasterio so I'm wondering if this is still actually a problem and hopefully an easy fix.
Expected behavior and actual behavior.
I expected/hoped to run rasterio with numpy 2.
Steps to reproduce the problem.
Install numpy nightly wheels and rasterio from github main/master. You'll get an error something like:
import rasterio
../../../miniconda3/envs/test-environment/lib/python3.11/site-packages/rasterio/__init__.py:27: in <module>
from rasterio._vsiopener import _opener_registration
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E ImportError: numpy.core.multiarray failed to import (auto-generated because you didn't call 'numpy.import_array()' after cimporting numpy; use '<void>numpy._import_array' to disable if you are certain you don't need it).
rasterio/_vsiopener.pyx:1: ImportError
During handling of the above exception, another exception occurred:
tmp_path = PosixPath('/tmp/pytest-of-runner/pytest-0/test_netcdf_grid_mapping_round0')
def test_netcdf_grid_mapping_round_trip(tmp_path):
ds = cf_0gm_no_coords()
new_crs = CRS.from_epsg(4326)
assert ds.geo.crs is None
new_ds = ds.geo.write_crs(new_crs, inplace=False)
assert new_ds.geo.crs == new_crs
nc_out = tmp_path / "test.nc"
new_ds.to_netcdf(nc_out)
assert nc_out.is_file()
> nc_ds = xr.open_dataset(nc_out, decode_coords="all")
geoxarray/tests/test_accessor/test_accessor_dataset.py:117:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../miniconda3/envs/test-environment/lib/python3.11/site-packages/xarray/backends/api.py:553: in open_dataset
engine = plugins.guess_engine(filename_or_obj)
../../../miniconda3/envs/test-environment/lib/python3.11/site-packages/xarray/backends/plugins.py:150: in guess_engine
engines = list_engines()
../../../miniconda3/envs/test-environment/lib/python3.11/site-packages/xarray/backends/plugins.py:139: in list_engines
return build_engines(entrypoints)
../../../miniconda3/envs/test-environment/lib/python3.11/site-packages/xarray/backends/plugins.py:112: in build_engines
external_backend_entrypoints = backends_dict_from_pkg(entrypoints_unique)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
entrypoints = [EntryPoint(name='rasterio', value='rioxarray.xarray_plugin:RasterioBackend', group='xarray.backends')]
def backends_dict_from_pkg(
entrypoints: list[EntryPoint],
) -> dict[str, type[BackendEntrypoint]]:
backend_entrypoints = {}
for entrypoint in entrypoints:
name = entrypoint.name
try:
backend = entrypoint.load()
backend_entrypoints[name] = backend
except Exception as ex:
> warnings.warn(f"Engine {name!r} loading failed:\n{ex}", RuntimeWarning)
E RuntimeWarning: Engine 'rasterio' loading failed:
E numpy.core.multiarray failed to import (auto-generated because you didn't call 'numpy.import_array()' after cimporting numpy; use '<void>numpy._import_array' to disable if you are certain you don't need it).
Environment Information
- rasterio version (
python -c "import rasterio; print(rasterio.__version__)"): github main - GDAL version (
python -c "import rasterio; print(rasterio.__gdal_version__)"): 3.8.3 - Python version (
python -c "import sys; print(sys.version.replace('\n', ' '))"): Python 3.11 - Operation System Information (
python -c "import platform; print(platform.platform())"): Github actions ubuntu-latest
Full log: https://github.com/geoxarray/geoxarray/actions/runs/7866956375/job/21461938995?pr=63
Installation Method
See CI job link above:
conda uninstall --force-remove -y pyarrow;
python -m pip install --extra-index-url https://pypi.fury.io/arrow-nightlies/ --prefer-binary --pre pyarrow;
python -m pip install \
--index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple/ \
--trusted-host pypi.anaconda.org \
--no-deps --pre --upgrade \
matplotlib \
numpy \
pandas \
scipy; \
python -m pip install \
--no-deps --upgrade \
git+https://github.com/dask/dask \
git+https://github.com/dask/distributed \
git+https://github.com/zarr-developers/zarr \
git+https://github.com/Unidata/cftime \
git+https://github.com/mapbox/rasterio \
git+https://github.com/pydata/bottleneck \
git+https://github.com/pydata/xarray;