Tags: angelayi/ao
Tags
update test-infra to release version (pytorch#1391) * update test-infra to release version Summary: pytorch/test-infra#6016 landed recently which is breaking our ROCm builds We point to a special branch of test-infra created just before this PR to unblock the v0.7.0 release. Test Plan: CI Reviewers: Subscribers: Tasks: Tags: * Update .github/workflows/build_wheels_linux.yml --------- Co-authored-by: Andrey Talman <atalman@fb.com>
Add TTFT benchmarks + update sparsity benchmarks (pytorch#1140) This PR adds in TTFT token benchmarks to torchAO, and also updates the benchmarking script to handle sparsity a bit nicer + use the 2:4 sparse checkpoints that are available. Additionally also adds in padding support for int8 dynamic quant + 2:4 sparsity, which we were missing before.
Add TTFT benchmarks + update sparsity benchmarks (pytorch#1140) This PR adds in TTFT token benchmarks to torchAO, and also updates the benchmarking script to handle sparsity a bit nicer + use the 2:4 sparse checkpoints that are available. Additionally also adds in padding support for int8 dynamic quant + 2:4 sparsity, which we were missing before.
check `scale.ndim` before applying `t`/`transpose` (pytorch#1339) * check `scale.ndim` before applying `t`/`transpose` because (a) `scale` could be 0D/1D and `transpose` and (b) the args and kwargs of `torch.ops.aten.transpose.int` would supply `dim0` and `dim1`, leading to cause dim canonicalization to fail. e.g. [`torch._prims_common.canonicalize_dims`](https://github.com/pytorch/pytorch/blob/07906f2/torch/_prims_common/__init__.py#L704) Signed-off-by: Masaki Kozuki <mkozuki@nvidia.com> * add test of `.t()` and `.transpose(0, 1)` Signed-off-by: Masaki Kozuki <mkozuki@nvidia.com> * change cond to transpose scale Signed-off-by: Masaki Kozuki <mkozuki@nvidia.com> --------- Signed-off-by: Masaki Kozuki <mkozuki@nvidia.com>
Fix 20x slowdown of FP6 kernel due to device properties query (pytorc… …h#1128) Fix 20x slowdown of FP6 kernel due to device properties query (pytorch#1092) Replace `cudaGetDeviceProperties` with `cudaDeviceGetAttribute` Co-authored-by: Tobias van der Werff <33268192+tobiasvanderwerff@users.noreply.github.com>
Fix 20x slowdown of FP6 kernel due to device properties query (pytorc… …h#1128) Fix 20x slowdown of FP6 kernel due to device properties query (pytorch#1092) Replace `cudaGetDeviceProperties` with `cudaDeviceGetAttribute` Co-authored-by: Tobias van der Werff <33268192+tobiasvanderwerff@users.noreply.github.com>
PreviousNext