New conan graph explain command to search, compare and explain missing binaries by memsharded · Pull Request #14694 · conan-io/conan

memsharded · 2023-09-07T13:35:39Z

Changelog: Feature: New conan graph explain command to search, compare and explain missing binaries.
Docs: conan-io/docs#3486
Close #14464
Close #14810
Close #4089
Close #6770

franramirez688

It's looking great! Only a few observations/doubts about the changes.

AbrilRBS · 2023-09-07T15:15:20Z

Pinging @danimtb which just spoke about this a few days ago

danimtb

I think it is fine for an initial implementation, although my main request would be to make it work with options as well. Also, as a nice feedback for the user, it would be awesome to output (maybe via stderr?) the closest match for the profile provided (but I guess that could be done in the future)

memsharded · 2023-09-11T11:15:28Z

I think it is fine for an initial implementation, although my main request would be to make it work with options as well. Also, as a nice feedback for the user, it would be awesome to output (maybe via stderr?) the closest match for the profile provided (but I guess that could be done in the future)

Sure, the options will be added, it was just the POC idea.

I'll check about the closest binary thing.

franramirez688 · 2023-11-27T15:37:24Z

@memsharded is this supposed to happen?

$ conan listx find-binaries "hell/1.0.0" -pr proof
Finding binaries in the cache
Finding binaries in remote conancenter
Local Cache
  hell/1.0.0
    hell/1.0.0#3f7c271fed307c6bee398d19e00f8b28 (2023-11-27 15:32:04 UTC)
      hell/1.0.0#3f7c271fed307c6bee398d19e00f8b28:40a4bd5c82fcddf7dcfbe92d65687290441859b6
        settings: Macos, x86_64, Release, apple-clang, 17, libc++, 15
        options: fPIC=True, shared=False
        diff
          settings: Debug
          explanation: This binary was built with different settings (compiler, build_type).
conancenter
  WARN: There are no matching recipe references

Where proof profile contains:

[options]
*:shared=False
[settings]
build_type=Debug

There are no binaries built with Debug mode, but it's showing me one binary with those diff points.

franramirez688 · 2023-11-27T15:39:40Z

@memsharded yes, I just saw some tests showing the closest binary found.

memsharded · 2023-11-27T15:57:12Z

Yes, the report is as follow:

The conan listx find-binaries report for each binary:

settings: Macos, x86_64, Release, apple-clang, 17, libc++, 15
options: fPIC=True, shared=False

This means this binary is built with Release.

Then the diff says:

diff
          settings: Debug
          explanation: This binary was built with different settings (compiler, build_type).

It means that you are asking for a Debug binary, but the found one is different in this setting.

…uture

franramirez688

This is great! 👏
I've only suggested one minor thing.

commit 98d8f6a Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Sun Dec 17 20:37:54 2023 +0100 Try to address NMakeDeps quoting issues (conan-io#15140) * Better spacing and quotes for defines in NMakeDeps * No need to escape parenthesis in defines * Fix tests * Follow proper syntax definition * Add numeric test * Ensure functional tests cover the quoting * Add test for space in values * fix values with whitespace in nmaketoolchain defines * fix nmakedeps and test * simplify * review --------- Co-authored-by: danimtb <danimanzaneque@gmail.com> commit 94a3549 Author: memsharded <james@conan.io> Date: Sat Dec 16 23:45:06 2023 +0100 fix broken test after source() VirtualBuildEnv commit 6958893 Author: James <james@conan.io> Date: Fri Dec 15 14:00:05 2023 +0100 Define toolset and architecture in CMakePresets.json for VS to avoid mismatches (conan-io#15215) * checking what breaks * fix test * testing it really fix something * wip * wip commit 7921be1 Author: James <james@conan.io> Date: Fri Dec 15 12:45:07 2023 +0100 adding tools.microsoft:winsdk_version conf (conan-io#15272) * adding tools.microsoft:winsdk_version conf * fix tests * wip * fix * review commit 018441d Author: James <james@conan.io> Date: Fri Dec 15 12:38:10 2023 +0100 new CMake.ctest() method (conan-io#15282) commit 053f22a Author: James <james@conan.io> Date: Fri Dec 15 12:23:57 2023 +0100 Feature/cmakedeps legacy adapter (conan-io#15207) * reusing fixture, will reduce CI time * proposal for legacy cmake-like generator * better ux for VS not found error (conan-io#15250) * better ux * improve * Update conan/tools/microsoft/visual.py Co-authored-by: Rubén Rincón Blanco <git@rinconblanco.es> --------- Co-authored-by: Rubén Rincón Blanco <git@rinconblanco.es> * fix and remove more xfails (conan-io#15251) fix and remove xfails * Fix/save restore portable (conan-io#15253) * make save-restore portable * msg * require-replaces proposal (conan-io#15136) * wip * test passing * wip * wip * wip * wip * wip * wip * new approach * extra checks * more tests * review * Fix failing tests in macos (conan-io#15255) Fix tests in macos * Add test to ensure &: syntax works in `--requires` install (conan-io#15258) Add test to ensure &: syntax qorks in --requires install * add CONAN_LOG_LEVEL env-var (conan-io#15263) * add CONAN_LOG_LEVEL env-var * Update conan/cli/command.py --------- Co-authored-by: Francisco Ramírez <franchuti688@gmail.com> * review * review --------- Co-authored-by: Rubén Rincón Blanco <git@rinconblanco.es> Co-authored-by: Rubén Rincón Blanco <rubenrb@jfrog.com> Co-authored-by: Francisco Ramírez <franchuti688@gmail.com> commit 9afacb4 Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Fri Dec 15 11:32:21 2023 +0100 Add different ref tracking syntax for `<host_version>` (conan-io#15274) * Use light=True where possible in build_requires tests * Sketch for allowing tracking of different ref * Add remote input to build-map command * Update conans/client/graph/graph_builder.py Co-authored-by: James <memsharded@gmail.com> * Update conans/client/graph/graph_builder.py Co-authored-by: James <memsharded@gmail.com> * Update conans/client/graph/graph_builder.py Co-authored-by: James <memsharded@gmail.com> * Fix tests --------- Co-authored-by: James <memsharded@gmail.com> commit 92af096 Author: Luis Caro Campos <3535649+jcar87@users.noreply.github.com> Date: Fri Dec 15 10:10:45 2023 +0000 Add pyproject.toml (conan-io#15186) * Move to pyproject.toml * Exclude conan server * Try adding a line * Upgrade pip in venv * update setuptools too * Read version from conans.__init__ * Restore setup.py for compatibility * Minimal pyproject toml * Restore setup.py * Restore jenkinsfile commit 0c786c5 Author: Carlos Zoido <mrgalleta@gmail.com> Date: Fri Dec 15 01:36:22 2023 +0100 Add environment information to CMakePresets (conan-io#15192) * build to config and run to test preset * first draft * revert * wip * fix format * test ctest * simplify * get env in right place * move test * fix windows * hopefully fix tests * fix test * fix test again * simplify test * add opt-out * wip * auto_generate for run env * minor changes commit 5becb2a Author: Juan <35701596+juansblanco@users.noreply.github.com> Date: Thu Dec 14 15:58:32 2023 +0100 Make paths format in layout consistent on Windows (conan-io#15260) Changelog: Omit Docs: Omit commit 9f023c3 Author: James <james@conan.io> Date: Thu Dec 14 15:48:28 2023 +0100 proposal for source() buildenv discussion (conan-io#15153) * proposal for source() buildenv discussion * wip, failing * trying alternative opt-in class attribute * fix * wip * wip * add cmake_layout test * wip * named auto_generate commit 0843372 Author: James <james@conan.io> Date: Thu Dec 14 12:34:54 2023 +0100 fix migration lru (conan-io#15135) * fix migration lru * fix back migration commit c655809 Author: Carlos Zoido <mrgalleta@gmail.com> Date: Wed Dec 13 16:32:02 2023 +0100 Add pkglist formatter for conan export command (conan-io#15266) add pkglist formatter and test commit d0e6a3f Author: James <james@conan.io> Date: Wed Dec 13 13:59:50 2023 +0100 optimize test (conan-io#15214) * optimize test * fixes * fix commit 6f3f445 Author: James <james@conan.io> Date: Wed Dec 13 12:57:00 2023 +0100 add CONAN_LOG_LEVEL env-var (conan-io#15263) * add CONAN_LOG_LEVEL env-var * Update conan/cli/command.py --------- Co-authored-by: Francisco Ramírez <franchuti688@gmail.com> commit 032eefc Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Tue Dec 12 19:43:09 2023 +0100 Add test to ensure &: syntax works in `--requires` install (conan-io#15258) Add test to ensure &: syntax qorks in --requires install commit 694b7d1 Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Tue Dec 12 14:57:14 2023 +0100 Fix failing tests in macos (conan-io#15255) Fix tests in macos commit 286d552 Author: James <james@conan.io> Date: Tue Dec 12 14:16:18 2023 +0100 require-replaces proposal (conan-io#15136) * wip * test passing * wip * wip * wip * wip * wip * wip * new approach * extra checks * more tests * review commit 7aa0b4b Author: James <james@conan.io> Date: Tue Dec 12 12:21:02 2023 +0100 Fix/save restore portable (conan-io#15253) * make save-restore portable * msg commit 1c43d5f Author: James <james@conan.io> Date: Tue Dec 12 09:43:07 2023 +0100 fix and remove more xfails (conan-io#15251) fix and remove xfails commit 2717bbe Author: James <james@conan.io> Date: Mon Dec 11 20:06:14 2023 +0100 better ux for VS not found error (conan-io#15250) * better ux * improve * Update conan/tools/microsoft/visual.py Co-authored-by: Rubén Rincón Blanco <git@rinconblanco.es> --------- Co-authored-by: Rubén Rincón Blanco <git@rinconblanco.es> commit 5ef6824 Author: James <james@conan.io> Date: Mon Dec 11 15:18:06 2023 +0100 drop --build unused patterns unused functionality (conan-io#15233) commit 1d058e9 Author: James <james@conan.io> Date: Mon Dec 11 13:47:21 2023 +0100 fix deploy absolute folders (conan-io#15244) * fix deploy absolute folders * fix tests commit 63ee9c7 Author: James <james@conan.io> Date: Mon Dec 11 07:59:37 2023 +0100 allow hard set_name() set_version() (conan-io#15211) commit 37573ee Author: James <james@conan.io> Date: Thu Dec 7 15:09:24 2023 +0100 Tests maintenance, fix some xfail, remove other xfails (conan-io#15234) remove xfail commit e6aa291 Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Thu Dec 7 15:04:23 2023 +0100 Add `in` operator support for ConanFile's `self.dependencies` (conan-io#15221) * Add in operator support for ConanFile self.dependencies * Ensure tool_requires also work commit a6f42bf Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Wed Dec 6 12:57:44 2023 +0100 Fix range escaping in conflict reports involving ranges (conan-io#15222) * Add failing test that should pass after issue is fixed * Fix range escaping in conflict reports involving ranges commit acbaf44 Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Mon Dec 4 23:16:38 2023 +0100 Fix `conan graph explain` binary distance calculation (conan-io#15212) * Bugfix binary distance calculation * Add proper distance culling test commit 5493a77 Author: James <james@conan.io> Date: Mon Dec 4 08:27:53 2023 +0100 reusing fixture, will reduce CI time (conan-io#15204) commit 412a027 Author: James <james@conan.io> Date: Fri Dec 1 13:58:04 2023 +0100 New [platform_requires] similar to system-tools, but for all requires (conan-io#14871) * wip * test passing * wip * wip * wip * wip * wip * renaming and review * fixes * add missing dumps * Simplified if-else clauses. Keeping more prio for platform_tool_requires * Renamed first naming proposal * review test, new check * fix test * fix --------- Co-authored-by: Francisco Ramirez de Anton <franchuti688@gmail.com> commit 1f548d8 Author: Luis Caro Campos <3535649+jcar87@users.noreply.github.com> Date: Fri Dec 1 09:57:43 2023 +0000 Fix pyinstaller script on Windows (conan-io#15194) commit dc98b6a Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Fri Dec 1 10:25:16 2023 +0100 Fix `conan graph info --format=html` reporting misleading conflicting nodes (conan-io#15196) * Fix graph info --format=html conflict report misleading nodes * Add node map test commit 3f3fd45 Author: memsharded <james@conan.io> Date: Thu Nov 30 00:33:54 2023 +0100 wip commit 9145028 Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Wed Nov 29 22:47:57 2023 +0100 Add `redirect_stdout` to CMake integration methods (conan-io#15150) * Add redirect_stdout to CMake integration methods * Fix windows tests * Simplify tests * Directly pass stdout & stderr streams instead of files * Ensure outputs are redirected to the correct streams * Fix unittests * Update conans/test/functional/toolchains/cmake/test_cmake_toolchain.py Co-authored-by: Carlos Zoido <mrgalleta@gmail.com> * Remove unused method * Add cmake.test stdout test, fix failing test * Simpify testing * Change testing approach for build * Change testing approach for build * Finally fix tests (hopefully) * Simplify tests * Comment wording * Check for emptyness in Windows --------- Co-authored-by: Carlos Zoido <mrgalleta@gmail.com> commit 71d863a Author: James <james@conan.io> Date: Wed Nov 29 17:17:48 2023 +0100 poc for filtering lists with profile-like inputs (conan-io#14694) * poc for filtering lists with profile-like inputs * per-package and options and distance * wip * wip * wip * refactor list format compact * wip * wip * ready * fix test * fix tests * converted to graph find-binaries approach * wip * review * Create custom formatters to ensure we can expand the command in the future * review --------- Co-authored-by: Rubén Rincón <rubenrb@jfrog.com> commit 25070c1 Author: Luis Caro Campos <3535649+jcar87@users.noreply.github.com> Date: Wed Nov 29 14:02:38 2023 +0000 Cleanup pyinstall.py (conan-io#15188) commit 29e9f95 Author: James <james@conan.io> Date: Wed Nov 29 10:58:08 2023 +0100 proposal for deploy() feature (conan-io#15172) * proposal for deploy() feature * rename argument --deploy-package * rename arg commit f0a1b35 Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Tue Nov 28 15:45:17 2023 +0100 Fix serialization of tool_requires in `conan profile show --format=json` (conan-io#15185) * Fix serialization of tool_requires in profile show --format=json * Serialize recipe reference as repr in profile tool_requires * Fix test commit a5605d1 Author: James <james@conan.io> Date: Tue Nov 28 14:29:54 2023 +0100 Refactor ``conan list --format=compact`` format code (conan-io#15177) * refactor list format compact * wip * wip commit 2e56813 Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Mon Nov 27 16:47:23 2023 +0100 Add test showing that `configure()` can be used to set `deprecated` (conan-io#15180) Add test showing that configure can set deprecated option just fin commit 65ec31c Author: Juan <35701596+juansblanco@users.noreply.github.com> Date: Sat Nov 25 08:01:42 2023 +0100 Removed conan migration compatibility warning (conan-io#15174) removed warning and related test commit dde76f6 Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Fri Nov 24 19:20:32 2023 +0100 Fix `conan graph info --format=text` being printed to stderr (conan-io#15170) Fix graph info --format=text being printed to stderr commit 0d9f52f Author: Rubén Rincón Blanco <rubenrb@jfrog.com> Date: Fri Nov 24 07:00:27 2023 +0100 Allow querying the contents of settings.yml (and settings_user!) from ConfigAPI (conan-io#15151) * Allow querying the contents of settings.yml (and settings_user!) from the API * Add tests * Pretty print settings, make ConfigAPI.builtin_confs a @Property * Fix tests, ensure null is properly printed * Remove conan config settings command until it's asked by someone * Remove unneeded test * Remove unused import

poc for filtering lists with profile-like inputs

0217710

memsharded added the type: look into label Sep 7, 2023

memsharded requested review from AbrilRBS, czoido and franramirez688 September 7, 2023 13:36

franramirez688 reviewed Sep 7, 2023

View reviewed changes

Comment thread conan/api/subapi/list.py Outdated

Comment thread conan/cli/commands/list.py Outdated

Comment thread conans/test/integration/command_v2/list_test.py Outdated

danimtb reviewed Sep 11, 2023

View reviewed changes

memsharded added this to the 2.0.12 milestone Sep 11, 2023

memsharded added 4 commits September 20, 2023 00:17

Merge branch 'release/2.0' into feature/list_filter_profile

48a1ddf

per-package and options and distance

16c76f1

Merge branch 'release/2.0' into feature/list_filter_profile

bc079c8

wip

dcdb0f7

memsharded mentioned this pull request Sep 24, 2023

[feature] A friendlier output when pre-built packages are missing (precisely listing the config diffs). #14810

Closed

1 task

czoido modified the milestones: 2.0.12, 2.0.13, 2.0.14 Sep 26, 2023

memsharded modified the milestones: 2.0.14, 2.0.15 Oct 26, 2023

memsharded added 7 commits November 24, 2023 17:32

wip

fec08a1

wip

27fa5f5

wip

79eb03a

refactor list format compact

4071934

wip

106deff

wip

135bf0f

wip

16acdaa

memsharded mentioned this pull request Nov 26, 2023

Refactor conan list --format=compact format code #15177

Merged

memsharded added 2 commits November 27, 2023 00:57

fix test

0342868

fix tests

5d6d682

memsharded requested a review from franramirez688 November 27, 2023 09:47

memsharded assigned AbrilRBS Nov 27, 2023

memsharded added 5 commits November 27, 2023 23:49

converted to graph find-binaries approach

e840cc8

merged release/2.0 and move functionality from cmd to conan_api.list

613ded5

wip

9417b38

review

0167034

Merge branch 'release/2.0' into feature/list_filter_profile

85886c8

memsharded marked this pull request as ready for review November 28, 2023 16:58

memsharded added the type: feature label Nov 28, 2023

AbrilRBS approved these changes Nov 29, 2023

View reviewed changes

Create custom formatters to ensure we can expand the command in the f…

48e5f2a

…uture

AbrilRBS approved these changes Nov 29, 2023

View reviewed changes

franramirez688 approved these changes Nov 29, 2023

View reviewed changes

Comment thread conan/cli/commands/graph.py Outdated

czoido approved these changes Nov 29, 2023

View reviewed changes

review

d3f4514

franramirez688 approved these changes Nov 29, 2023

View reviewed changes

memsharded merged commit 71d863a into conan-io:release/2.0 Nov 29, 2023

memsharded deleted the feature/list_filter_profile branch November 29, 2023 16:17

memsharded mentioned this pull request Nov 29, 2023

[question] filtering packages with "conan list" based on an option not part of the package id #14464

Closed

1 task

memsharded changed the title ~~poc for filtering lists with profile-like inputs~~ New conan graph explain command to search, compare and explain missing binaries Nov 29, 2023

This was referenced Nov 29, 2023

[feature request] Explain why a match was not found (conan why) #4089

Closed

[feature] Add "Best Matches" Binary Search Mode #6770

Closed

enable query for a package / profile combination, local and/or remote #6842

Closed

danimtb mentioned this pull request Dec 18, 2023

Conan graph explain docs conan-io/docs#3486

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New conan graph explain command to search, compare and explain missing binaries#14694

New conan graph explain command to search, compare and explain missing binaries#14694
memsharded merged 23 commits into
conan-io:release/2.0from
memsharded:feature/list_filter_profile

memsharded commented Sep 7, 2023 •

edited by czoido

Loading

Uh oh!

franramirez688 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AbrilRBS commented Sep 7, 2023

Uh oh!

danimtb left a comment

Uh oh!

memsharded commented Sep 11, 2023

Uh oh!

franramirez688 commented Nov 27, 2023 •

edited

Loading

Uh oh!

franramirez688 commented Nov 27, 2023

Uh oh!

memsharded commented Nov 27, 2023

Uh oh!

franramirez688 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

memsharded commented Sep 7, 2023 • edited by czoido Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

franramirez688 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AbrilRBS commented Sep 7, 2023

Uh oh!

danimtb left a comment

Choose a reason for hiding this comment

Uh oh!

memsharded commented Sep 11, 2023

Uh oh!

franramirez688 commented Nov 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

franramirez688 commented Nov 27, 2023

Uh oh!

memsharded commented Nov 27, 2023

Uh oh!

franramirez688 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

memsharded commented Sep 7, 2023 •

edited by czoido

Loading

franramirez688 commented Nov 27, 2023 •

edited

Loading