Repository navigation
Remove setuptools requirements in pip itself #1422
Description
Activity
+INT_MAX. As the maintainer the Chef tooling for pip I have huge issues every time pip changes its setuptools requirements, and the result is usually a non-functional system as you can't use pip to install the correct versions of things once it is broken.
there is no PEP426/MEBs yet, so unclear what your plan is for this?
I mean that pip can depend on setuptools for actually executing a setup.py (e.g. just assume that the setup.py will be using setuptools), but that pip itself no longer depends on setuptools. This is important for Wheel installs in particular.
right, but how are you planning on implementing this idea of "pip can depend on setuptools for actually executing a setup.py". daniel's "autosetuptools", was one idea, where we vendor pkg_resources and we automatically install setuptools when we need to, but that has concerns.
I don't know that we actually need to do anything. Pip will still require you to have setuptools installed in order to install from sdists. So for most people there will still be a "dependency" on setuptools. The difference being we've firewalled it off to only the setup.py execution so we don't run into problems where a broken setuptools means a broken pip where you have to manually install setuptools to fix it.
So, what I understand you to be saying is that we vendor pkg_resources, and redirect all of our internal calls to pkg_resources to the vendored version. We could go further and remove/migrate some of those calls to avoid pkg_resources (possibly in favour of the already-vendored distlib) but this is optional, and probably not worth the risk.
Users still need setuptools installed to install from sdist (because we inject setuptools into the setup.py invocation) but that's orthogonal to our internal use of pkg_resources.
Is that a correct interpretation?
If so, +1 from me.
Yup that's basically what I mean.
Maybe we should consider this for 1.5.1. There has been a number of folks grumbling about the setuptools 0.8 requirement such as http://stackoverflow.com/questions/20905350/latest-pip-fails-with-requires-setuptools-0-8-for-dist-info
yea, I guess more people than we might have imagined are using a mixture of old setuptools/distribute + new pip.
although in the SO, the answer given is fine. the author thinking he has setuptools 2.0.2 must be for another python.I just did a very simple proof of concept patch (replace "import pkg_resources" with "from pip._vendor import pkg_resources") and it seems to work for some basic tests.
Also, with this patch, you can (almost!) create a standalone zipfile of pip which can be run from a bare Python interpreter. There's a problem with finding the SSL certificate (see below) but that can probably be sorted with a bit of effort.
>.\ww\Scripts\python.exe .\pip.zip install setuptools pip Downloading/unpacking setuptools Could not fetch URL https://pypi.python.org/simple/setuptools/: There was a problem confirming the ssl certificate: [Errno 2] No such file or directoryThe following
__main__.pyfixes the SSL issue, but vendoring setuptools is harder than the proof of concept above (no surprise :-)). I'm putting a copy of my__main__.pyhere anyway in case it's worth adding to pip as part of this change.import sys, os from pip.runner import run from pkgutil import get_data from tempfile import TemporaryDirectory if __name__ == '__main__': cert_data = get_data('pip._vendor.requests', 'cacert.pem') with TemporaryDirectory() as td: tempname = os.path.join(td, 'cacert.pem') with open(tempname, 'wb') as tf: tf.write(cert_data) os.environ['REQUESTS_CA_BUNDLE'] = tempname exit = run() if exit: sys.exit(exit)Right on. When I did it, it was pretty much as simple as "from pip.vendor import pkg_resources". This takes care of pip's need to enumerate installed packages.
I also remember a special case or two intended to help with upgrading distribute or setuptools that kindof breaks here.
On the other hand setuptools is only needed by the setup.py sub-processes and pip shouldn't vendor it.
It will be more complicated to fix pip to get smarter about setup/build dependencies but that is a totally different issue. I will be very happy when "pip install setuptools" always works.
pkg_resources also uses _markerlib, so that needs to be vendored, and pkg_resources amended, as well.
I think pkg_resources also has its own markers implementation, used in a different place, that came later. But vendoring pkg_resources' vendored markerlib is fairly easy. See old patch :-)
Moving this to 1.5.1
$ python -m pip Usage: /Users/dstufft/.virtualenvs/650292f54bdb66c8/bin/python -m pip <command> [options] Commands: install Install packages. uninstall Uninstall packages. freeze Output installed packages in requirements format. list List installed packages. show Show information about installed packages. search Search PyPI for packages. wheel Build wheels from your requirements. zip Zip individual packages. unzip Unzip individual packages. bundle Create pybundles. help Show help for commands. General Options: -h, --help Show help. -v, --verbose Give more output. Option is additive, and can be used up to 3 times. -V, --version Show version and exit. -q, --quiet Give less output. --log-file <path> Path to a verbose non-appending log, that only logs failures. This log is active by default at /Users/dstufft/.pip/pip.log. --log <path> Path to a verbose appending log. This log is inactive by default. --proxy <proxy> Specify a proxy in the form [user:passwd@]proxy.server:port. --timeout <sec> Set the socket timeout (default 15.0 seconds). --exists-action <action> Default action when a path already exists: (s)witch, (i)gnore, (w)ipe, (b)ackup. --cert <path> Path to alternate CA bundle.$ python -m pip install requests Downloading/unpacking requests Using download cache from /Users/dstufft/.pip/cache/https%3A%2F%2Fpreview-pypi.python.org%2Fpackages%2F2.7%2Fr%2Frequests%2Frequests-2.1.0-py2.py3-none-any.whl Installing collected packages: requests Successfully installed requests Cleaning up...$ python -m pip list pip (1.5, /Users/dstufft/projects/pip) requests (2.1.0) wsgiref (0.1.2)The above are in a virtualenv where I've uninstalled setuptools. I had to use
python -m pipbecause I installed using setup.py instead of Wheel so the generated pip script depended on pkg_resources (which isn't the case for Wheel installs).Here's what happens when you force it to use a sdist:
$ python -m pip install --no-use-wheel requests Downloading/unpacking requests Using download cache from /Users/dstufft/.pip/cache/https%3A%2F%2Fpreview-pypi.python.org%2Fpackages%2Fsource%2Fr%2Frequests%2Frequests-2.1.0.tar.gz Running setup.py (path:/Users/dstufft/.virtualenvs/650292f54bdb66c8/build/requests/setup.py) egg_info for package requests Traceback (most recent call last): File "<string>", line 3, in <module> ImportError: No module named setuptools.command Complete output from command python setup.py egg_info: Traceback (most recent call last): File "<string>", line 3, in <module> ImportError: No module named setuptools.command ---------------------------------------- Cleaning up... Command python setup.py egg_info failed with error code 1 in /Users/dstufft/.virtualenvs/650292f54bdb66c8/build/requests Storing debug log for failure in /Users/dstufft/.pip/pip.logI'll probably explicitly check for setuptools to be installed and give a better error message when installing from sdist, but it's working.
Nice. Can you put this up as a PR when it's ready, and I'll have a play with it?
Yup!
- added 2 commits that reference this issue
on Jan 9, 2014 - locked as resolved and limited conversation to collaborators
on Jun 5, 2019
The past two released of pip have had issues relating to the setuptools dependency. Ideally pip itself will no longer have any sort of dependency on setuptools, and the only actual dependency for setuptools will exist when a project uses setup.py.