Back out "Fixing a bug where allocating a 4GB block results in using 8GB of memory (#95827)" #96796

jaewonlee-fb · 2023-03-15T03:11:07Z

Summary:
Original commit changeset: a19273017a2a

Original Phabricator Diff: D43969564

Test Plan: unlandayc

Reviewed By: terrycsy

Differential Revision: D44080273

pytorch-bot · 2023-03-15T03:11:09Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/96796

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit fcfe9dc:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

linux-foundation-easycla · 2023-03-15T03:11:11Z

The committers listed above are authorized under a signed CLA.

✅ login: jaewonlee-fb (9b6040530c4b8ee4ee2fc5b57e526faddb84e67a)

facebook-github-bot · 2023-03-15T03:11:44Z

This pull request was exported from Phabricator. Differential Revision: D44080273

facebook-github-bot · 2023-03-15T03:18:02Z

This pull request was exported from Phabricator. Differential Revision: D44080273

facebook-github-bot · 2023-03-15T04:47:35Z

This pull request was exported from Phabricator. Differential Revision: D44080273

facebook-github-bot · 2023-03-15T04:56:37Z

This pull request was exported from Phabricator. Differential Revision: D44080273

facebook-github-bot · 2023-03-15T05:04:07Z

This pull request was exported from Phabricator. Differential Revision: D44080273

facebook-github-bot · 2023-03-15T05:11:31Z

This pull request was exported from Phabricator. Differential Revision: D44080273

facebook-github-bot · 2023-03-15T16:41:42Z

This pull request was exported from Phabricator. Differential Revision: D44080273

ngimel · 2023-03-15T16:41:44Z

cc @akamali, looks like 32MB caching threshold is too aggressive - can you make the thresholds configurable via env vars, similarly to how it's done for cuda caching allocators?

facebook-github-bot · 2023-03-15T16:50:24Z

This pull request was exported from Phabricator. Differential Revision: D44080273

…8GB of memory (pytorch#95827)" (pytorch#96796) Summary: Pull Request resolved: pytorch#96796 Original commit changeset: a19273017a2a Original Phabricator Diff: D43969564 Parameters set too aggressively hurting some critical use cases. We should revisit this idea with more rigorous testing -- at least configurable parameters (use GFLAGs?) with conservative default values (e.g., INF for pinned memory caching size limit). ----------------------------------------------------------------------------------------------------------------------- Test Plan: This is an unland diff. unlandayc Reviewed By: terrycsy, banitag1, yuchenhao, liangluofb Differential Revision: D44080273 fbshipit-source-id: ce587db7db3db4bd0ebcabdcba4c3b39fd2424c0

jianyuh

I will stamp to unblock and then let @akamali / @jaewonlee-fb to figure out more proper approach to land the original PR.

jaewonlee-fb · 2023-03-15T17:49:42Z

@pytorchbot merge

pytorchmergebot · 2023-03-15T17:51:31Z

Merge failed

Reason: This PR needs a label
If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Details for Dev Infra team

Raised by workflow job

jaewonlee-fb · 2023-03-15T23:23:52Z

@pytorchbot merge

pytorchmergebot · 2023-03-15T23:25:40Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2023-03-16T00:01:10Z

Merge failed

Reason: 1 jobs have failed, first few of them are: trunk / win-vs2019-cuda11.7-py3 / test (default, 3, 5, windows.g5.4xlarge.nvidia.gpu)

Details for Dev Infra team

Raised by workflow job

facebook-github-bot · 2023-03-16T05:33:51Z

This pull request was exported from Phabricator. Differential Revision: D44080273

…8GB of memory (pytorch#95827)" (pytorch#96796) Summary: Pull Request resolved: pytorch#96796 Original commit changeset: a19273017a2a Original Phabricator Diff: D43969564 Parameters set too aggressively hurting some critical use cases. We should revisit this idea with more rigorous testing -- at least configurable parameters (use GFLAGs?) with conservative default values (e.g., INF for pinned memory caching size limit). ----------------------------------------------------------------------------------------------------------------------- Test Plan: This is an unland diff. unlandayc Reviewed By: terrycsy, jianyuh, banitag1, yuchenhao, liangluofb Differential Revision: D44080273 fbshipit-source-id: 7743a8d53e5fdd11aa72843802e2abc31585211a

facebook-github-bot · 2023-03-16T05:42:07Z

This pull request was exported from Phabricator. Differential Revision: D44080273

jaewonlee-fb · 2023-03-16T16:33:39Z

@pytorchbot merge

pytorchmergebot · 2023-03-16T16:35:28Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

…8GB of memory (#95827)" (#96796) Summary: Original commit changeset: a19273017a2a Original Phabricator Diff: D43969564 ----------------------------------------------------------------------------------------------------------------------- Test Plan: unlandayc Reviewed By: terrycsy Differential Revision: D44080273 Pull Request resolved: pytorch/pytorch#96796 Approved by: https://github.com/jianyuh, https://github.com/davidberard98

facebook-github-bot added the fb-exported label Mar 15, 2023

jaewonlee-fb force-pushed the export-D44080273 branch 2 times, most recently from 1b84c0a to 109b4b2 Compare March 15, 2023 04:47

jaewonlee-fb force-pushed the export-D44080273 branch from 109b4b2 to 9b60405 Compare March 15, 2023 04:56

jaewonlee-fb force-pushed the export-D44080273 branch from 9b60405 to def63c7 Compare March 15, 2023 05:04

jaewonlee-fb force-pushed the export-D44080273 branch from def63c7 to 59045a2 Compare March 15, 2023 05:11

jaewonlee-fb force-pushed the export-D44080273 branch from 59045a2 to b0fbd4b Compare March 15, 2023 16:41

jaewonlee-fb force-pushed the export-D44080273 branch from b0fbd4b to 9e4ca4a Compare March 15, 2023 16:50

jianyuh approved these changes Mar 15, 2023

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Mar 15, 2023

jianyuh added the release notes: cuda release notes category label Mar 15, 2023

davidberard98 self-requested a review March 15, 2023 22:18

davidberard98 approved these changes Mar 15, 2023

View reviewed changes

jaewonlee-fb force-pushed the export-D44080273 branch from 9e4ca4a to bf2de70 Compare March 16, 2023 05:33

jaewonlee-fb force-pushed the export-D44080273 branch from bf2de70 to fcfe9dc Compare March 16, 2023 05:42

pytorchmergebot added the Merged label Mar 16, 2023

pytorchmergebot closed this in aeb3db8 Mar 16, 2023

Back out "Fixing a bug where allocating a 4GB block results in using 8GB of memory (#95827)" #96796

Back out "Fixing a bug where allocating a 4GB block results in using 8GB of memory (#95827)" #96796

Uh oh!

Conversation

jaewonlee-fb commented Mar 15, 2023

Uh oh!

pytorch-bot bot commented Mar 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/96796

✅ No Failures

Uh oh!

linux-foundation-easycla bot commented Mar 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Mar 15, 2023

Uh oh!

facebook-github-bot commented Mar 15, 2023

Uh oh!

facebook-github-bot commented Mar 15, 2023

Uh oh!

facebook-github-bot commented Mar 15, 2023

Uh oh!

facebook-github-bot commented Mar 15, 2023

Uh oh!

facebook-github-bot commented Mar 15, 2023

Uh oh!

facebook-github-bot commented Mar 15, 2023

Uh oh!

ngimel commented Mar 15, 2023

Uh oh!

facebook-github-bot commented Mar 15, 2023

Uh oh!

jianyuh left a comment

Choose a reason for hiding this comment

Uh oh!

jaewonlee-fb commented Mar 15, 2023

Uh oh!

pytorchmergebot commented Mar 15, 2023

Merge failed

Uh oh!

jaewonlee-fb commented Mar 15, 2023

Uh oh!

pytorchmergebot commented Mar 15, 2023

Merge started

Uh oh!

pytorchmergebot commented Mar 16, 2023

Merge failed

Uh oh!

facebook-github-bot commented Mar 16, 2023

Uh oh!

facebook-github-bot commented Mar 16, 2023

Uh oh!

jaewonlee-fb commented Mar 16, 2023

Uh oh!

pytorchmergebot commented Mar 16, 2023

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

pytorch-bot bot commented Mar 15, 2023 •

edited

Loading

linux-foundation-easycla bot commented Mar 15, 2023 •

edited

Loading