[FSDP][optim_state_dict] Print out more useful error message for opti…

…m_state_dict

Summary: Print out more useful error message for optim_state_dict

Test Plan: CI

Reviewed By: wz337

Differential Revision: D43556073

fbshipit-source-id: b365b95263ad304781b792b643d3156495554eac

Merge branch 'master' into whc/fix_ci

Back out "Fixing a bug where allocating a 4GB block results in using …

…8GB of memory (pytorch#95827)" (pytorch#96796)

Summary:
Pull Request resolved: pytorch#96796

Original commit changeset: a19273017a2a

Original Phabricator Diff: D43969564

Parameters set too aggressively hurting some critical use cases. We should revisit this idea with more rigorous testing -- at least configurable parameters (use GFLAGs?) with conservative default values (e.g., INF for pinned memory caching size limit).

-----------------------------------------------------------------------------------------------------------------------

Test Plan:
This is an unland diff.

unlandayc

Reviewed By: terrycsy, banitag1, yuchenhao, liangluofb

Differential Revision: D44080273

fbshipit-source-id: ce587db7db3db4bd0ebcabdcba4c3b39fd2424c0

Update on "[Testing] Flip Switch"

cc soumith voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 desertfire

[ghstack-poisoned]

Update on "[WIP] Extend aot autograd dedup guards to params, stop usi…

…ng positions"

cc soumith penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx desertfire

[ghstack-poisoned]

Update on "[inductor] Refactor memory management code in wrapper code…

…gen"


Summary: use inheritance to simplify CppWrapperCodeGen and to prepare for AOT codegen

cc soumith voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10

[ghstack-poisoned]

Update on "[primTorch] handle out in `sort` meta function"

cc soumith voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx desertfire

[ghstack-poisoned]

Remove temp breakage

Fix CPU bitwise shifts for out-of-limit shift values

Negative shift values and positive shift values greater than the
bit size of the dtype (limit 0..bits) now yield expected results
which are consistent with numpy.

Left shift with an out-of-limit shift value result in a value of 0.
Right shift with an out-of-limit shift value results in a value of -1
for negative inputs and 0 for non-negative inputs (sign preserving).

Fixes pytorch#70904

Signed-off-by: BJ Hargrave <hargrave@us.ibm.com>

Update on "Default specialize_int to False"

Signed-off-by: Edward Z. Yang <ezyangmeta.com>

cc soumith voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 desertfire

[ghstack-poisoned]