Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v0.4.14 seems to have resolver issues #7606

Open
stas00 opened this issue Sep 20, 2024 · 5 comments
Open

v0.4.14 seems to have resolver issues #7606

stas00 opened this issue Sep 20, 2024 · 5 comments

Comments

@stas00
Copy link

stas00 commented Sep 20, 2024 •
edited
Loading

our uv pip compile -v setup.py --extra replug has just started failing on CI once uv has been updated:

  × No solution found when resolving dependencies:
  ╰─▶ Because nmslib==2.1.1 depends on numpy>=1.10.0,<1.17 and only nmslib<=2.1.1 is available, we can conclude that nmslib>=2.1.1
      depends on numpy>=1.10.0,<1.17.
      And because pyserini==0.23.0 depends on numpy>=1.18.1, we can conclude that nmslib>=2.1.1 and pyserini==0.23.0 are incompatible.
      And because pyserini==0.23.0 depends on nmslib>=2.1.1 and dawn depends on pyserini==0.23.0, we can conclude that your requirements
      are unsatisfiable.

The parser seems to be getting the requirements wrong, because:

$ git clone https://github.com/nmslib
$ cd nmslib
$ git checkout v2.1.1
$ grep -Ir "<1\.17"
python_bindings/setup.py:dep_list.append("numpy>=1.10.0,<1.17 ; python_version=='2.7'")

but I'm running py3.10

an easier way to repro is to just run:

$ uv pip install nmslib==2.1.1
Resolved 4 packages in 152ms
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: numpy==1.16.6
  Caused by: Build backend failed to build wheel through `build_wheel()` with exit status: 1
--- stdout:

--- stderr:
Running from numpy source directory.
/data/env/cache/uv/sdists-v4/pypi/numpy/1.16.6/YYsNUdkR_MaadjuXSLKgQ/numpy-1.16.6.zip/numpy/distutils/misc_util.py:476: SyntaxWarning: "is" with a literal. Did you mean "=="?
  return is_string(s) and ('*' in s or '?' is s)
Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "/env/cache/uv/builds-v0/.tmpEg6RuR/lib/python3.10/site-packages/setuptools/build_meta.py", line 421, in build_wheel
    return self._build_with_temp_dir(
  File "/env/cache/uv/builds-v0/.tmpEg6RuR/lib/python3.10/site-packages/setuptools/build_meta.py", line 403, in _build_with_temp_dir
    self.run_setup()
  File "/env/cache/uv/builds-v0/.tmpEg6RuR/lib/python3.10/site-packages/setuptools/build_meta.py", line 503, in run_setup
    super().run_setup(setup_script=setup_script)
  File "/env/cache/uv/builds-v0/.tmpEg6RuR/lib/python3.10/site-packages/setuptools/build_meta.py", line 318, in run_setup
    exec(code, locals())
  File "<string>", line 419, in <module>
  File "<string>", line 398, in setup_package
  File "/data/env/cache/uv/sdists-v4/pypi/numpy/1.16.6/YYsNUdkR_MaadjuXSLKgQ/numpy-1.16.6.zip/numpy/distutils/core.py", line 26, in <module>
    from numpy.distutils.command import config, config_compiler, \
  File "/data/env/cache/uv/sdists-v4/pypi/numpy/1.16.6/YYsNUdkR_MaadjuXSLKgQ/numpy-1.16.6.zip/numpy/distutils/command/config.py", line 19, in <module>
    from numpy.distutils.mingw32ccompiler import generate_manifest
  File "/data/env/cache/uv/sdists-v4/pypi/numpy/1.16.6/YYsNUdkR_MaadjuXSLKgQ/numpy-1.16.6.zip/numpy/distutils/mingw32ccompiler.py", line 34, in <module>
    from distutils.msvccompiler import get_build_version as get_build_msvc_version
ModuleNotFoundError: No module named 'distutils.msvccompiler'
---
@stas00
Copy link
Author

stas00 commented Sep 20, 2024 •
edited
Loading

0.4.13 also has an issue

uv pip install uv==0.4.13
uv pip install nmslib==2.1.1

same error as the end of the OP

I didn't check them all but uv==0.4.0 works fine.

The correct resolution should be numpy>=1.10.0 for py3.10 for nmslib==2.1.1

$ uv pip install uv==0.4.0
Resolved 1 package in 122ms
Prepared 1 package in 396ms
Uninstalled 1 package in 51ms
Installed 1 package in 125ms
 - uv==0.4.13
 + uv==0.4.0

$ uv pip install  nmslib==2.1.1
Resolved 4 packages in 322ms
Installed 1 package in 1.05s
 + nmslib==2.1.1

@stas00
Copy link
Author

stas00 commented Sep 20, 2024

The actual dependency I need to satisfy is pyserini==0.23.0, which pulls in nmslib.

robinjhuang added a commit to Comfy-Org/comfy-cli that referenced this issue Sep 21, 2024
@robinjhuang
Restrict uv version to 0.4.0
6299e12 
Latest versions have resolver issues.

astral-sh/uv#7606
@zanieb
Copy link
Member

zanieb commented Sep 21, 2024

Sorry struggling to reproduce this, am I doing something wrong?

❯ echo "nmslib==2.1.1" | uvx uv@0.4.14 pip compile - --python-platform linux --python 3.10
Resolved 4 packages in 3ms
# This file was autogenerated by uv via the following command:
#    uv pip compile - --python-platform linux --python 3.10
nmslib==2.1.1
numpy==2.1.1
    # via nmslib
psutil==6.0.0
    # via nmslib
pybind11==2.6.1
    # via nmslib

@stas00
Copy link
Author

stas00 commented Sep 21, 2024 •
edited
Loading

indeed, seems to be fine alone:

$ echo "nmslib==2.1.1" | uv pip compile - --python-platform linux --python 3.10
Resolved 4 packages in 245ms
# This file was autogenerated by uv via the following command:
#    uv pip compile - --python-platform linux --python 3.10
nmslib==2.1.1
numpy==1.16.6
    # via nmslib
psutil==6.0.0
    # via nmslib
pybind11==2.6.1
    # via nmslib

now let's try with the actual package that I depend on:

$ echo "pyserini==0.23.0" | uv pip compile - --python-platform linux --python 3.10
  × No solution found when resolving dependencies:
  ╰─▶ Because nmslib==2.1.1 depends on numpy>=1.10.0,<1.17 and only nmslib<=2.1.1 is available, we can conclude that nmslib>=2.1.1 depends
      on numpy>=1.10.0,<1.17.
      And because pyserini==0.23.0 depends on numpy>=1.18.1, we can conclude that nmslib>=2.1.1 and pyserini==0.23.0 are incompatible.
      And because pyserini==0.23.0 depends on nmslib>=2.1.1 and you require pyserini==0.23.0, we can conclude that your requirements are
      unsatisfiable.

that numpy>=1.10.0,<1.17 dependency is the trigger, and it should be numpy>=1.10.0 instead.

By looking at the source code of these repos - the only suspect to where <1.17 may have come from I think is:

$ git clone https://github.com/nmslib
$ cd nmslib
$ git checkout v2.1.1
$ grep -Ir "<1\.17"
python_bindings/setup.py:dep_list.append("numpy>=1.10.0,<1.17 ; python_version=='2.7'")

@zanieb
Copy link
Member

zanieb commented Sep 21, 2024

Looks like this regressed in 0.4.13

❯ echo "pyserini==0.23.0" | uvx uv@0.4.13 pip compile - --python-platform linux --python 3.10
Installed 1 package in 6ms
  × No solution found when resolving dependencies:
  ╰─▶ Because nmslib==2.1.1 depends on numpy>=1.10.0,<1.17 and only nmslib<=2.1.1 is available, we can conclude that nmslib>=2.1.1 depends on
      numpy>=1.10.0,<1.17.
      And because pyserini==0.23.0 depends on numpy>=1.18.1, we can conclude that nmslib>=2.1.1 and pyserini==0.23.0 are incompatible.
      And because pyserini==0.23.0 depends on nmslib>=2.1.1 and you require pyserini==0.23.0, we can conclude that your requirements are unsatisfiable.
      
❯ echo "pyserini==0.23.0" | uvx uv@0.4.12 pip compile - --python-platform linux --python 3.10
Resolved 83 packages in 27ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@zanieb @stas00