Repository navigation
Relaxing / Ignoring constraints during dependency resolution #8076
Description
Activity
- changed the title
[-]a --whitelist feature for the New resolver ? a pipdeptree like description of dependancies ? a gui over that?[/-][+]a --whitelist feature for the New resolver ? a pipdeptree-like description of dependancies ? a gui over that?[/+]on Apr 18, 2020 - changed the title
[-]a --whitelist feature for the New resolver ? a pipdeptree-like description of dependancies ? a gui over that?[/-][+]a --relax feature for the New resolver ? a pipdeptree-like description of dependancies ? a gui over that?[/+]on Apr 18, 2020 @stonebig Would you mind separate the other wishes part into their own issues? It would be much easier to discuss them that way.
As for relxing, I am honestly uncomfortable with having such a impactful feature handy for the general audience. I was by chance also having a similar discussion in the Pipenv tracker, and both the Pipenv one and your are exactly situations I personally think the maintainers have a valid point to restrict the versions, and a user should not be able to override them easily. It’s still useful to have this option somewhere since indeed there are packages with incorrect metadata out there, but pip is too low in the packaging management realm to implement the feature IMO.
Ok, separating other whishes in a few minutes.
this is moved on another issue:- having the beautifull "pipdeptree" features in the standard pip: . a beautifull description of what package needs (or is needed) by what package of what version, . the possibility to get that programatically as json answers.- changed the title
[-]a --relax feature for the New resolver ? a pipdeptree-like description of dependancies ? a gui over that?[/-][+]Relaxing / Ignoring constraints during dependnecy resolution[/+]on Apr 18, 2020 - changed the title
[-]Relaxing / Ignoring constraints during dependnecy resolution[/-][+]Relaxing / Ignoring constraints during dependency resolution[/+]on Apr 18, 2020 Thanks for filing this @stonebig! I've gone ahead and re-titled this to issue to be more clearly scoped.
We have seen multiple groups of users express interest in a feature like this. @pfmoore @uranusjr and I have had this come up in our discussions during our work on the resolver, and we are aware of this user need.
We don't know how exactly this would work and what approach we'd be taking here -- we're gonna visit this specific topic at a later date, once the new resolver implementation is at feature parity with the current resolver.
a basic GUI on Pip (tkinter or web) would still be nice, to have a better view of all the coming version conflicts.
This is a completely separate request, and can be built outside of pip and doesn't need to be built into pip. If someone wants to build this outside of pip and later propose bringing it into pip (with clear reasoning for why it can't live outside pip), that'd be perfect. I don't think pip's maintainers are going to be developing/integrating this into pip, and I welcome others to try to build such tooling on-top-of or outside of pip.
I think there has been a "pip GUI" project undertaken as part of IDLE in the past, but I don't have the time to take a look right now. :)
Building a dsitribution WinPython is quite simple:
- download in a dedicated directory all the wheels (and version) you want,
- then pip install -r requirement.txt
- then one by one, try to fix all the problems:
- missing wheels,
- pip download --dest,
- or cgohlke site of wonders
- or github/gitlab (/pip-forge one day ?)
- non-existing wheels,
- do-compile-yourself (often fails, like for cartopy, or for Python-recent version)
- raise issues to package maintainer
- wheels whose beloved version of dependancies mutualy contradicts
- ask maintainer to relax or upgrade his/her dependancies (very slow process)
- recompile it yourself without the annoying constraint,
- go back in version to an older one (with potential security or known issues fixed since ages)
- or drop the wheel.
- missing wheels,
I dream of a way to reverse the problem:
- showing package maintainer how their 'too restrictive' constraints makes them incompatible with the rest of the world, (hence a GUI or a pypi website feature ?):
- give your requirements.txt
- precise your "beloved" package,
- the site/gui tells you what fits / what contradicts / what downgrade your package imposes
- or do a third kind of constraints on dependancies in wheel specification:
- supported constraints (you can speak of a problem to the maintainers when you have this "set"),
- a "support_requires" next to "install_requires" and "extra_requires" ?
Reacted by Frederik Rietdijk121 remaining items
+1
Very often the project will still work, even if the exact dependency version is different.We already have the great constraints file, where we can manually specify version of dependencies (
-coption). Can we add an option to for constraints to override the package-specified version, eg--force-constraints? So similar to @notatallshaw proposal, but reusing the existing constraint infra.For example, assume that package
Arequiresnumpy==1.23and packageBrequiresnumpy<1.20. Even if we find that packageBworks well with Numpy 1.23 (at least for the set of features that we care about), we will have a hard time installing things withpip, we would need to use workarounds like--no-deps, and manually installing deps later.With
--force-constraintsoption we would simply specify a constraintnumpy==1.23and we are good, a warning can still be logged during installation, that the dependency version has been overwritten by the const.Implementation would be super simple - just override the dependency version requirement if it conflicts with the constraint, and log a warning.
Reacted by Leonardo Fontoura, Erik Brendel, Maurits van Rees, Will Thompson, Dan Miller, 152334H, Arya Massarat, Grégory Starck, Sage Betko and Kalin StoyanovThis proposal is great. If this proposal can be finished in a way that PyPI maintainers are willing to accept, I would love to discuss how we (@NabuCasa / @home-assistant) can sponsor someone to build this feature. Prefer to sponsor a maintainer or active contributor to pip.
Reacted by Frederik RietdijkI started on a PR to fix this issue. I think my code works for the problems described here. My use case is a bit different though, so I won't finish the PR and have just closed it. But I would be happy if someone picks up my work in their own PR.
The summary is in this comment: #11537 (comment)
Let me post the todo items from there.- Confirmation from others in the issue that this indeed fixes their problem. I will add a comment there pointing to this PR.
- A decision on the format: extend the constraints syntax with -o like in this PR, or use a separate overrides file with strictly limited formatting. Looks like the discussion is leaning towards the last.
- tests
- documentation
The first item is something anyone can help with: try out my PR and see if that fixes your problem or not, and report it back.
Reacted by Tzu-ping Chung and Frederik RietdijkReacted by KOLANICH@mauritsvanrees I like #11537 and while I have not tested it, I think it solves our issues.
I agree with @pfmoore that adding this type of syntax adds more complexity and that a separate overrides file is preferable, at least for now.
In the future I can imagine a single new style file but that should really be out of scope here. Alternatively, I could imagine a new operator (version specifier?) for this,
=o, somypkg =o 0.1.5to override all constraints and set a version (or set no version), but this might warrant a PEP; we definitely would not want individual packages to use this operator.Reacted by KOLANICHIn #9948, we're discussing adding a per-requirement
--no-depsto allow users to opt-out of dependency handling for certain entries in a requirements.txt file. While that doesn't solve the "relaxing" part of the request here, it does provide ignoring functionality -- in a highly pip-specific manner.If the proposed solution there works for you, please leave a 👍🏽 reaction on this comment or let us know on that issue why your usecase isn't something workable with that.
Here's a "bigger picture" question that I briefly mentioned above but I think should be made explicit. Are we willing to support people using pip in already-broken environments?
Suppose for example, we get a bug report "
pip install foofails with an error". No-one can reproduce the issue locally. After much investigation, it turns out that the user is installing into an existing environment that has a horribly broken mess of pre-installed packages, including a number of packages deep in the dependency tree offoo. But the user has done nothing but use supported pip features (like this one) to create that environment.I'm fine with us saying we won't support that. And I'd be happy with having a requirement that we can only investigate bugs reported with either a reproducer that starts from an empty environment, or where the user has explicitly confirmed that
pip checkshows that their environment is clean. But do we want to offer tools that make it easier for users to get into that situation in the first place? There's still a maintenance cost involved, as we sometimes spend time trying to understand the user's problem before we get round to asking the "obvious questions".I'm sure that the people with use cases who need this functionality would use it appropriately and with care (no sarcasm here, genuinely). It's the people who find advice on the internet and use it inappropriately without understanding the implication ("just use
--break-system-packages, it fixes your issue"... 🙁) who could easily find something like this a big footgun.Are we willing to support people using pip in already-broken environments?
by broken do you mean
pip checkfails ? I guess..but actually, isn't it already the case ipso-facto (that pip running/installing in already "broken" env is allowed/supported) ?
I mean it's not like, for instance, debian apt that refuses to continue installing new packages if one(or more) of them are actually "broken" and not fixed (in some sense of "broken", as far as I know).
After much investigation, it turns out that the user is installing into an existing environment that has a horribly broken mess of pre-installed packages,
I would guess/hope that someone would ask to
pip listorpip checkthe given user installation/environment. Or simply/directly request him to try with fresh/clean python (v)environment too.isn't it already the case ipso-facto (that pip running/installing in already "broken" env is allowed/supported) ?
No. There are many reasons why pip doesn't run
pip checkbefore allowing an install, but that doesn't mean that it's supported. Think of it like "undefined behaviour". You can do it, and it might allow you do achieve some really neat goal, but that doesn't mean you can rely on it. Unless we (the pip maintainers) decide between us that we want to support it, in the sense of applying our stability and backward compatibility policies to it, for example.I would guess/hope that someone would ask to pip list or pip check the given user installation/environment.
It's surprisingly uncommon that we ask that. To an extent that's what I'm saying, do we want to start with "please run
pip check", with the implication that if it reports problems, we'll say "that's your issue, fix that and you'll be fine"?If a pip environment is already broken, people may need tools to unbreak it. At the minimum, if
pip checkcurrently fails,pip installshould still work, otherwise you have no other option but to throw away your environment.I think it should work like this, and I have no idea if that is the case now:
pip install good combination-> workspip check-> passespip install bad combination: doespip checkafter analysing the dependencies, which fails, so the installation is stopped before anything is actually installed.- Now somehow break the environment.
pip check-> failspip install another good combination-> This should work. Yes,pip checkfails after analysing the dependencies, but it should realise thatpip checkalready fails in the current environment, sopip installshould continue, as the user may be trying a first step to unbreak the environment.
That said: I think it is fair to update the bug report issue template to ask people to include the output of
pip check.Reacted by Will ThompsonIt's a while since I checked the details, but from what I recall,
pip installruns apip checkwhen it's complete, and reports if there is an issue. At that point,pip uninstallandpip installstill work fine. We don't ever fail because of a broken environment, you're just in "you broke it, you get to fix it" territory.There are plenty of ways you can use documented pip features to break an environment. The obvious trivial one is
pip install --no-deps Awhen A depends on B. Orpip install A; pip uninstall B. But that doesn't (IMO) mean we're under an obligation to do anything sane when working with that environment (it's hard even to be sure what "sane" means!) I think you can reasonably expect thatpip uninstallandpip install --no-depswork. Beyond that, by all means give it a go, but you may hit issues. I think you'll find that plenty more than that will work reasonably well, but I'm not guaranteeing anything.To be clear, I'm talking here about "broken" in the sense of "some dependency metadata isn't satisfied". You can break environments in other ways (manually delete files, for example) that will leave pip unusable. You're on your own in that case.
Reacted by Greg Roodt- Reacted by Edgar Ramírez Mondragón and Jalin Wang
Interesting, at least for the:
- speed,
- compatibility aim with pypa,
- carbon footprint
- extra.
Adding the pipdeptree style features would be great, especially if accessible by api/not using the command line
Small update for anyone who needs this feature, uv has a pip-like install interface and supports a
--overrideflag.uv pipinterface has a useful behavior when dealing with conflicting dependencies. If packages a and b have incompatible dependencies, runninguv pip install a bwill fail due to the conflict. However, installing them one by one (uv pip install a, thenuv pip install b) will incur automatically uninstalling or upgrading conflicting versions. This makes it available to manually adjust/override dependencies — meanwhilepipjust refuses me again and again :(This seems to be an old pip behavior and sometime can lead to problems. But a flag ( e.g., --ignore-conflicts ) to enbale this behavior manually definitely helps a lot when encountering such a situtation.
If packages a and b have incompatible dependencies, running
uv pip install a bwill fail due to the conflict. However, installing them one by one (uv pip install a, thenuv pip install b) will incur automatically uninstalling or upgrading conflicting versions. This makes it available to manually adjust/override dependenciesThis behavior was copied from pip, although the exact resolution might not be the same between pip and uv pip you should see the same general behavior.
meanwhile
pipjust refuses me again and again :(This seems to be an old pip behavior and sometime can lead to problems. But a flag ( e.g., --ignore-conflicts ) to enbale this behavior manually definitely helps a lot when encountering such a situtation.
I'm not sure what you mean, uv doesn't ignore the conflicts of a given install request, same as pip, maybe you're looking for the
--upgradeflag which eagerly upgrades rather than trying to keep the version installed?If you have a specific problem where
pip install afollowed bypip install brefuses to install and you think it should then please create a new issue with a reproducible example.- marked Unpin versions of dependencies as a consnumer #13699 as a duplicate of this issue
on Dec 14, 2025
What's the problem this feature will solve?
Puting together some packages that have by default incompatible constraints.
Indeed:
. "accepting complains on this known/focus set of package",
. you're on your own support if you deviate, but not necessarly bad.
==> The new resolver may create more problems than solutions, when trying to build an environment with a large set of package.
Describe the solution you'd like
Be able to ignore voluntary some constraints
Wish:
pip install Spyder --relax relaxrules.r, with relax file below meaning:. if you want PyQt5, it must be 5.14.x
. if you want Jedi, it must be >=0.16
Alternative Solutions
Today:
Additional context
Maintaining WinPython
** Current pip check **
** Current workaround **
. PyQt5-5.14.2 (as pip doesn't have a "long term support" of PyQt5-5.12, so the fresher version is safer)
other wishes: