[dynamo][invoke_subgraph] Input aliasing and mutation check in Dynamo #148953

anijain2305 · 2025-03-11T07:05:41Z

Stack from ghstack (oldest at bottom):

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @amjames

[ghstack-poisoned]

pytorch-bot · 2025-03-11T07:05:45Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/148953

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 4 Unrelated Failures

As of commit 6420cb4 with merge base ce54c43 ():

NEW FAILURES - The following jobs have failed:

linux-binary-libtorch-release / libtorch-cpu-shared-with-deps-release-build / build (gh)
Process completed with exit code 1.
linux-binary-manywheel / manywheel-py3_9-cuda11_8-build / build (gh)
Process completed with exit code 1.
linux-binary-manywheel / manywheel-py3_9-cuda12_6-build / build (gh)
Process completed with exit code 1.
linux-binary-manywheel / manywheel-py3_9-cuda12_8-build / build (gh)
Process completed with exit code 1.

UNSTABLE - The following jobs are marked as unstable, possibly due to flakiness on trunk:

inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (cpu_inductor_torchbench, 1, 2, linux.8xlarge.amx) (gh) (#149977)
detectron2_fcos_r_50_fpn
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (dynamic_cpu_inductor_torchbench, 1, 2, linux.8xlarge.amx) (gh) (#149979)
detectron2_fcos_r_50_fpn
pull / cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) (gh) (#149370)
REGRESSION: benchmark ('aotdispatcher_training_subclass_cpu', 'compile_time_instruction_count') failed, actual result 9987075560 is 1.60% higher than expected 9830000000 ±+1.50% if this is an expected regression, please update the expected results.
pull / linux-jammy-py3-clang12-executorch / test (executorch, 1, 1, lf.ephemeral.linux.2xlarge) (gh) (#144480)
backends/xnnpack/test/passes/test_convert_to_linear.py::TestConvertToLinear::test_fp32_convert_to_linear

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: e77333d Pull Request resolved: #148953

anijain2305

This does not work if a free variable is mutated.

Update - This works now.

…aliasing" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

ghstack-source-id: ffc382a Pull Request resolved: #148953 [dynamo][invoke_subgraph] Faster aliasing checks ghstack-source-id: ffc382a Pull Request resolved: #148997

…k in Dynamo" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

zou3519 · 2025-03-27T13:59:32Z

torch/_dynamo/output_graph.py

+        # Stores the versions of the input tensors at the time they are inserted
+        # as placeholders in the graph. This is used to track input mutation.
+        self._input_versions_at_beginning: list[int] = []


Can you assert that inference_mode is off here (during Subtracer construction)

zou3519 · 2025-03-27T14:02:42Z

torch/_dynamo/output_graph.py

+                        msg = f"Output-to-output aliasing detected at nodes {output_storages[storage]} and {out_node}"
+                        return AliasingInfo(True, msg)
+                    output_storages[storage] = out_node
+


Can the outputs ever be List[Tensor]? If not, let's assert against it.
(I hope they cannot be List[Tensor], I would want speculate_subgraph to flatten the outputs)

Done. Added a test as well.

torch/_dynamo/output_graph.py

test/higher_order_ops/test_invoke_subgraph.py

zou3519

LGTM, but have some minor comments, please read.

…k in Dynamo" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

anijain2305 · 2025-03-28T01:42:46Z

@pytorchbot merge

pytorchmergebot · 2025-03-28T01:44:40Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2025-03-28T02:00:53Z

Merge failed

Reason: 3 jobs have failed, first few of them are: linux-binary-libtorch-release / libtorch-cpu-shared-with-deps-release-build / build, linux-binary-manywheel / manywheel-py3_9-cuda11_8-build / build, linux-binary-manywheel / manywheel-py3_9-cuda12_6-build / build

Details for Dev Infra team

Raised by workflow job

anijain2305 · 2025-03-28T03:42:08Z

@pytorchbot merge -i

pytorchmergebot · 2025-03-28T03:44:15Z

Merge started

Your change will be merged while ignoring the following 8 checks: pull / cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu), pull / linux-jammy-py3-clang12-executorch / test (executorch, 1, 1, lf.ephemeral.linux.2xlarge), inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (cpu_inductor_torchbench, 1, 2, linux.8xlarge.amx), inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (dynamic_cpu_inductor_torchbench, 1, 2, linux.8xlarge.amx), linux-binary-libtorch-release / libtorch-cpu-shared-with-deps-release-build / build, linux-binary-manywheel / manywheel-py3_9-cuda11_8-build / build, linux-binary-manywheel / manywheel-py3_9-cuda12_8-build / build, linux-binary-manywheel / manywheel-py3_9-cuda12_6-build / build

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Pull Request resolved: #150090 Approved by: https://github.com/eellison, https://github.com/zou3519 ghstack dependencies: #149087, #149667, #150036, #148953

…pytorch#148953) Pull Request resolved: pytorch#148953 Approved by: https://github.com/zou3519 ghstack dependencies: pytorch#149087, pytorch#149667, pytorch#150036

Pull Request resolved: pytorch#150090 Approved by: https://github.com/eellison, https://github.com/zou3519 ghstack dependencies: pytorch#149087, pytorch#149667, pytorch#150036, pytorch#148953

ghstack-source-id: 4d85ff1 Pull Request resolved: pytorch/pytorch#148953 [dynamo][invoke_subgraph] Faster aliasing checks ghstack-source-id: 4d85ff1 Pull Request resolved: pytorch/pytorch#148997

[dynamo][invoke_subgraph] Use versions to check for input aliasing

4230b66

[ghstack-poisoned]

anijain2305 requested a review from zou3519 as a code owner March 11, 2025 07:05

pytorch-bot bot added ciflow/inductor module: dynamo labels Mar 11, 2025

anijain2305 added a commit that referenced this pull request Mar 11, 2025

[dynamo][invoke_subgraph] Use versions to check for input aliasing

4f01e76

ghstack-source-id: e77333d Pull Request resolved: #148953

anijain2305 added the topic: not user facing topic category label Mar 11, 2025

anijain2305 commented Mar 11, 2025

View reviewed changes

anijain2305 removed the request for review from zou3519 March 11, 2025 17:55

Update on "[dynamo][invoke_subgraph] Use versions to check for input …

411c214

…aliasing" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

Update on "[dynamo][invoke_subgraph] Use versions to check for input …

719aa8b

…aliasing" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

Update on "[dynamo][invoke_subgraph] Use versions to check for input …

5b36285

…aliasing" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

anijain2305 mentioned this pull request Mar 11, 2025

[dynamo][invoke_subgraph] Faster aliasing checks #148997

Closed

Update on "[dynamo][invoke_subgraph] Use versions to check for input …

519c0c5

…aliasing" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

anijain2305 mentioned this pull request Mar 12, 2025

[compile] Switch off inference_mode for fake prop while compiling #149072

Closed

anijain2305 added 5 commits March 12, 2025 14:46

Update on "[dynamo][invoke_subgraph] Use versions to check for input …

c44688c

…aliasing" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

Update on "[dynamo][invoke_subgraph] Use versions to check for input …

e143788

…aliasing" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

Update on "[dynamo][invoke_subgraph] Use versions to check for input …

98575f5

…aliasing" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

Update on "[dynamo][invoke_subgraph] Use versions to check for input …

742b5cd

…aliasing" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

Update on "[dynamo][invoke_subgraph] Use versions to check for input …

55fb0b1

…aliasing" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

anijain2305 changed the title ~~[dynamo][invoke_subgraph] Use versions to check for input aliasing~~ [dynamo][invoke_subgraph] Input aliasing and mutation check in Dynamo Mar 14, 2025

Update on "[dynamo][invoke_subgraph] Input aliasing and mutation chec…

e2e980c

…k in Dynamo" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

Update on "[dynamo][invoke_subgraph] Input aliasing and mutation chec…

f49b1a4

…k in Dynamo" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

Update on "[dynamo][invoke_subgraph] Input aliasing and mutation chec…

87b83c3

…k in Dynamo" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

Update on "[dynamo][invoke_subgraph] Input aliasing and mutation chec…

7aed2b3

…k in Dynamo" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

This was referenced Mar 27, 2025

[invoke_subgraph] Support None in the fwd output #150082

Closed

[inductor] No type promotion for slice_scatter #150090

Closed

zou3519 reviewed Mar 27, 2025

View reviewed changes

torch/_dynamo/output_graph.py Show resolved Hide resolved

zou3519 reviewed Mar 27, 2025

View reviewed changes

test/higher_order_ops/test_invoke_subgraph.py Show resolved Hide resolved

zou3519 approved these changes Mar 27, 2025

View reviewed changes

bohnstingl mentioned this pull request Mar 27, 2025

[HOP] Mutation and alias rework #146658

Closed

anijain2305 added 2 commits March 27, 2025 11:02

Update on "[dynamo][invoke_subgraph] Input aliasing and mutation chec…

a942f6f

…k in Dynamo" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

Update on "[dynamo][invoke_subgraph] Input aliasing and mutation chec…

6420cb4

…k in Dynamo" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Mar 28, 2025

pytorchmergebot added the merging label Mar 28, 2025

pytorchmergebot removed the merging label Mar 28, 2025

pytorchmergebot added the merging label Mar 28, 2025

pytorchmergebot added the Merged label Mar 28, 2025

pytorchmergebot closed this in c9ebf51 Mar 28, 2025

pytorchmergebot removed the merging label Mar 28, 2025

github-actions bot deleted the gh/anijain2305/697/head branch May 2, 2025 02:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[dynamo][invoke_subgraph] Input aliasing and mutation check in Dynamo #148953

[dynamo][invoke_subgraph] Input aliasing and mutation check in Dynamo #148953

Uh oh!

anijain2305 commented Mar 11, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Mar 11, 2025 •

edited

Loading

Uh oh!

anijain2305 left a comment •

edited

Loading

Uh oh!

zou3519 Mar 27, 2025 •

edited

Loading

Uh oh!

anijain2305 Mar 27, 2025

Uh oh!

zou3519 Mar 27, 2025 •

edited

Loading

Uh oh!

anijain2305 Mar 27, 2025

Uh oh!

Uh oh!

Uh oh!

zou3519 left a comment

Uh oh!

anijain2305 commented Mar 28, 2025

Uh oh!

pytorchmergebot commented Mar 28, 2025

Uh oh!

pytorchmergebot commented Mar 28, 2025

Uh oh!

anijain2305 commented Mar 28, 2025

Uh oh!

pytorchmergebot commented Mar 28, 2025

Uh oh!

Uh oh!

[dynamo][invoke_subgraph] Input aliasing and mutation check in Dynamo #148953

[dynamo][invoke_subgraph] Input aliasing and mutation check in Dynamo #148953

Uh oh!

Conversation

anijain2305 commented Mar 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Mar 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/148953

❌ 4 New Failures, 4 Unrelated Failures

Uh oh!

anijain2305 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zou3519 Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anijain2305 Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

zou3519 Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anijain2305 Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

zou3519 left a comment

Choose a reason for hiding this comment

Uh oh!

anijain2305 commented Mar 28, 2025

Uh oh!

pytorchmergebot commented Mar 28, 2025

Merge started

Uh oh!

pytorchmergebot commented Mar 28, 2025

Merge failed

Uh oh!

anijain2305 commented Mar 28, 2025

Uh oh!

pytorchmergebot commented Mar 28, 2025

Merge started

Uh oh!

Uh oh!

anijain2305 commented Mar 11, 2025 •

edited

Loading

pytorch-bot bot commented Mar 11, 2025 •

edited

Loading

anijain2305 left a comment •

edited

Loading

zou3519 Mar 27, 2025 •

edited

Loading

zou3519 Mar 27, 2025 •

edited

Loading