Repository navigation
Flag to allow uv pip install to install into specific environments (that do not necessarily contain uv) #1396
Description
Activity
I did also encounter this when trying to use uv as a drop in replacement for my pip projects.
Definitely agree, this is fundamental to allow a single UV installation to be used as a real standalone binary!
Yes we should support this. Is there a consensus on the name of the flag in other tools?
pip has the
--require-virtualenvoption which makes it behave like uv:$ pip install --require-virtualenv attrs ERROR: Could not find an activated virtualenv (required).but
--no-require-virtualenvseems too long of a name for an option in uv. And sure the default uv behavior is much saner default!pip uses
--prefixand--python(I think the difference relates to which Python pip should be run with, which doesn't matter to uv).
pypa/installer uses--prefix.I'm not too familiar with pdm, but the
--venvflag several of its command support is a name not a path (which is a good choice for high level workflow tool).Meta since I got confused:
- There's maybe some overlap between this issue and Cannot
uv pip installwithout virtualenv #1374 and Add flag to allow global installs e.g. for CI #1526 (which are duplicates of each other). In both cases the workaround is to setVIRTUAL_ENVexplicitly. - I view this issue as low priority and just providing a flag that does a very mechanical equivalent of
VIRTUAL_ENV=path uv pip install ...viauv pip install --prefix=path ... - The other issues is a flag that a) does something sort of like
VIRTUAL_ENV=$(/path/to/base/python -c "import sys; print(sys.prefix)") uv pip install ...viauv pip install --system ..., b) promises that setting that flag will make things work for a base Python, since it's not clear how intentional it is that settingVIRTUAL_ENVcurrently lets you point to a base Python. Those issues also maybe need more thinking about how to identify the right base Python, e.g. if you dopython -m uv ...should it look atsys.base_prefix?
- There's maybe some overlap between this issue and Cannot
pip uses
--prefixand--python(I think the difference relates to which Python pip should be run with, which doesn't matter to uv).A few things:
-
The
--pythonflag is desirable here because the tool that creates an environment is not necessarily the same program that uses an environment (perhaps in IDE). In this case, there is no knowledge of the base Python that was used and therefore no way to get information about its version, architecture, etc. It's not true to say that pip uses the flag to decide where to run (it's Python so it's already running) but rather it's entirely about information gathering.There is no way to avoid invoking the interpreter to get that data until PEP 739 is accepted (and implemented by various tools) and then another of the same kind for virtual environments is accepted (and implemented by various tools).
-
The
VIRTUAL_ENVworkaround indeed does not work well for base installations because the directory structure is not guaranteed and is often different based on the platform. For example, on Windows installations usually have apython.exeat the root whereas a virtual environment will have that under theScriptsdirectory. That command doesn't work in the official Python Docker image for instance:❯ docker run --rm python:3.12 python -c "import sys; print(sys.prefix)" /usr/local
Reacted by 0ptimista-
@ofek no, pip will literally run itself in a subprocess using a different interpreter with
--python: https://github.com/pypa/pip/blob/ebe491a82a13e6610697b22be00db363fb5ff5e3/src/pip/_internal/cli/main_parser.py#L82Thanks for linking the code, I didn't realize that's how they chose to implement it! My point still stands in that pip is already running and therefore the previous assertion that it needs to know where to run is not true (except for that implementation detail) but its purpose is rather to get details about the interpreter. Other tools like virtualenv, Hatch and Poetry use a script that returns the environment of a given interpreter:
Sorry, I'm still a little bit confused about what your ask is. Note that
uvdoes invoke the interpreter it is trying to install into in all cases:This issue really should just be adding a flag that does what the env var already does
I don't know if uv handles installing into base python on Windows correctly with the
VIRTUAL_ENVtrick (and that's more a question for #1374). But in the official docker image case you talk about uv does the right thing with the trick, I'm not sure why you say it doesn't work:docker run --rm python:3.12 bash -c 'pip install uv mypy; VIRTUAL_ENV=$(python -c "import sys; print(sys.prefix)") uv pip freeze'-
My point is that if we are provided an option for an explicit path then it would be far more rational for that option to be a Python binary which is ultimately invoked rather than a directory that must have a specific structure in order to find the aforementioned binary. The example I gave was a real one in which IDEs act upon paths to interpreters rather than directories per se.
I didn't know UV already polls the interpreter; that's good! So it looks like just an oversight to be able to control the specific Python interpreter.
-
What I was trying to show earlier was that the
VIRTUAL_ENVenvironment variable is working by happenstance because the defaultsys.prefixdirectory/usr/localhappens to have abindirectory which matches the expected Linux virtual environment structure. As you mentioned this doesn't work on Windows and I'm not certain that works for all Linux installations either.
-
Okay, I think I see! To summarise:
- Currently, you can do
VIRTUAL_ENV=/path/to/prefix uv pip ... - It would be nice if you could instead do
uv pip --prefix /path/to/prefix ...- This would make uv install into a specified virtual environment just a tiny bit nicer, which was my use case when opening this issue.
- Honestly, I don't even mind the env var that much, I think I just ran
uv pip install --helpand didn't see the pip option that I knew I could use to do this. My intention when opening issue was basically just a discoverability nit :-)
- Alternatively, the flag could be
uv pip --python-executable /path/to/python ... --python-executablemight have an advantage if you wanted this flag to be able to install into base environments, because while uv knows directory structure for venvs, it may not for base environments (see also Cannotuv pip installwithout virtualenv #1374 / Add flag to allow global installs e.g. for CI #1526 ). In particular, for base environment on Windows and for Linux distro provided Python (like when python is/usr/bin/python3and dist-packages nonsense is happening or whatever)
So I guess if I were uv, I'd figure out what/how I want to do 1374 / 1526 and maybe that determines what should happen here. Most of the demand for those issues is probably stuff like "make official python docker image work" or "make homebrew python work", for which logic equivalent to
VIRTUAL_ENV=$(python -c "import sys; print(sys.prefix)")works fine, modulo the easy to add extra Windows case. I spent some time looking at distro specific sysconfig stuff or framework builds or whatever else and frankly supporting those base environments looks horrible (or at least beyond my amateur interest. Also if PEP 668 is used widely, maybe easy to declare out of scope)- Currently, you can do
I'm hoping a solution to this would solve an edge case I'm running into. I often have a conda environment with nox that then creates virtualenvs. This leads to both
CONDA_PREFIXandVIRTUAL_ENVvariables being set. With pip, this is easy to work around usingpath/to/python -m pipto install packages in the correct environment (so #1632, which I think is intimately connected to this issue, would solve this). But having a flag like--python-executableor--prefixwould also solve this, as long as passing such a flag overridesCONDA_PREFIXandVIRTUAL_ENV. This would hopefully help wntrblm/nox#762 as well.I’m planning to work on this next week. I’ll post here with a concrete proposal once I’ve had time to internalize the comments. (Probably something like a
--pythonflag alongside a--systemflag but TBD.)Reacted by Shantanu, Michiel W. Beijen, T-256 and Sebastián RamírezReacted by Ofek Lev, Edgar Ramírez Mondragón, Oliver Mannion and Sebastián RamírezAs far as I can tell from reading the linked Poetry source, apart from adding the arguments to the CLI etc., the main thing that would need to change to support arbitrary Pythons (outside of a virtual environment) is that we'd need to call
sysconfig.get_paths()to get the various environment paths, rather than constructing them ourselves -- since the exact structure can vary for non-virtualenvs. Is there any reason this wouldn't work?Definitely some things to learn from in the Poetry source: python-poetry/poetry@e8f259a
sharing one use case of using
--prefixand--root,inside a multi stages docker build, one stage is building/collecting all packages and install them into a /build directory
and other stages just copy the content out of into their /usr/local folder.meanwhile I've worked around using those pip flags, by VIRTUAL_ENV=/user/local, and copy all of /user/local between the layers.
Reacted by Ron Nabuurs and Walker Philipssharing one use case of using
--prefixand--root,inside a multi stages docker build, one stage is building/collecting all packages and install them into a /build directory and other stages just copy the content out of into their /usr/local folder.
meanwhile I've worked around using those pip flags, by VIRTUAL_ENV=/user/local, and copy all of /user/local between the layers.
This is the exact use case I also have
Reacted by Israel Fruchter- added a commit that references this issue
on Feb 28, 2024 @charliermarsh I don't think the --python solved the equivalent
pip install --rootcase, you want us to open a new issue for that ?uv pip install --system uvCame to this looking for
--projectsupport foruv pip, which is documented as not supported byuv pip.
--pythonworks really well for this, though it's a bit more verbose, since required interpreter path instead of just project dir.
It looks like
uvalready supports this, I just need to lie aboutVIRTUAL_ENV.Looking for something like the equivalent of pip's
--prefixor--python.