Tags: keyan/torchrec

Update the tests (meta-pytorch#2521)

Summary:
Pull Request resolved: meta-pytorch#2521

We updated the `torch._dynamo.config.optimize_ddp` from `python_reducer` to `python_reducer_without_compiled_forward`, which will generate different number of compiled autograd graph. This is expected. We thus update the test.

Reviewed By: yf225

Differential Revision: D64815497

fbshipit-source-id: 3fbf6421f4855b41d7078118a64c6dd64ecc7682

add sharding_type argument to pipeline benchmark (meta-pytorch#2495)

Summary:
Pull Request resolved: meta-pytorch#2495

# context
* add sharding_type argument to the pipeline benchmark
* better control of different sharding types

Reviewed By: iamzainhuda

Differential Revision: D64676132

fbshipit-source-id: 8ceeba667b6b2f8aded3dcf3d894cf8cbca31d3e

Update actions/checkout 2

Fix release_build and gpu ci, use cu124 as default to be consistent with

torch

Update actions/checkout

Update actions/checkout 2

Migrate release_build to use pytorch test-infra linux_job.yml

Fix typo

Fix typo 2

CUDA 12.4 update as default for pypi and gpu ci

Add env

env variable pypi token

env variable pypi token 2

env variable pypi token 3

env variable pypi token 4

env variable pypi token 5

env variable pypi token 6

env variable pypi token 7

remove upgrade pip

remove upgrade pip 2

remove upgrade pip 3

add forward/backward test for _fbgemm_permute_pooled_embs (meta-pytor…

…ch#2480)

Summary:
Pull Request resolved: meta-pytorch#2480

# context
* S443491 is caused by using a customized [version](https://www.internalfb.com/code/fbsource/[552b2a3cb49a261daa48b68b3647e8a951a3aa1b]/fbcode/minimal_viable_ai/models/main_feed_mtml/pytorch_modules.py?lines=2610) (fb.permute_pooled_embs_auto_grad) of fbgemm.permute_pooled_embs_auto_grad
* the fb version doesn't dispatch to autograd but relied on a bug in fbgemm.permute_pooled_embs_auto_grad, which was fixed by D48574563
* The SEV was mitigated by switching to fbgemm version: D62040883
* this diff is to add more tests regarding fbgemm.permute_pooled_embs_auto_grad

# details
* `permute_pooled_embs_auto_grad` is called in `_fbgemm_permute_pooled_embs` function
* add forward and backward test for `_fbgemm_permute_pooled_embs` function

Reviewed By: ge0405

Differential Revision: D64195848

fbshipit-source-id: 237ad75028eb9583bb02a2f305defb083f0f280d

build-docs job builds from source (meta-pytorch#2466)

Summary:
Pull Request resolved: meta-pytorch#2466

Previously we build docs from nightly. Hard for us to see previews of docstring changes, here we change it to build from source.

Reviewed By: svekars

Differential Revision: D63784555

fbshipit-source-id: 2402fc7d31b91c9f1b7a503211c9d3e6b8f7641e

Support generating zero labels in unit tests (meta-pytorch#2463)

Summary:
Pull Request resolved: meta-pytorch#2463

Add an option `zero_labels` to generate zero labels instead of using random labels. When set to true, `rec_metric_value_test_launcher` will always compute the metric values using zero labels. If set to false (by default), it falls back to generating random labels by sampling uniformly between `[0, n_classes)`

This option is useful in scenarios where a unit test wants specifically use zero labels to test some behaviors. It is used in the subsequent D63751613 to surface unintended behavior due to tensor shapes computed based on dynamic inputs.

Reviewed By: borundev

Differential Revision: D63751613

fbshipit-source-id: d4072b69a771a146ffac4eaa7dbb289a9bb10b10

fix optimizer config to apply correctly in tests (meta-pytorch#2436)

Summary:
Pull Request resolved: meta-pytorch#2436

tsia, same as D63468297

Reviewed By: PaulZhang12

Differential Revision: D63555376

fbshipit-source-id: 9cd4721ef01c07a9b64420b0a1967adc0642dc0d

Revert D63150195: Multisect successfully blamed "D63150195: [o1][torc…

…hrec] setup new tensor weighted avg metric" for one test failure (meta-pytorch#2409)

Summary:
Pull Request resolved: meta-pytorch#2409

This diff reverts D63150195
D63150195: [o1][torchrec] setup new tensor weighted avg metric by billyang98 causes the following test failure:

Tests affected:
- *****

Here's the Multisect link:
https://www.internalfb.com/multisect/******
Here are the tasks that are relevant to this breakage:
T191382969: 50+ CI signals unhealthy for *****

The backout may land if someone accepts it.

If this diff has been generated in error, you can Commandeer and Abandon it.

Reviewed By: jasonzou0

Differential Revision: D63206118

fbshipit-source-id: c579f8deeec9b75ebb801e67cb1df41192877995

Add KT.regroup to kT pytree unflatten function to preserve the correc…

…t key order (meta-pytorch#2390)

Summary:
Pull Request resolved: meta-pytorch#2390

# context
* this diff basically re-order the keys in a KT every time after the flatten/unflatten step, based on the key order in the exported IR
* This diff is the short-term solution regarding to [post](https://fb.workplace.com/groups/1028545332188949/permalink/1042204770823005/)
 {F1865696002}

Reviewed By: PaulZhang12

Differential Revision: D59238744

fbshipit-source-id: 4d12f45cd6c056c88648aa5ba7b0295b55b1d581