I think I have found the underlying problem:

Problem

When the TT (and I assume any other) decomposition function calls truncated_svd (either through randomized_svd or directly), tensorly.svd dispatches it to torch.svd [^1], which (at least in my cases) dispatches it to the underlying cuSolver method gesvdj (cusolverDn<t>esvdj [^2], where <t> is the dtype).
The actual problem is that gesvdj apparently allocates full matrices[^3], even if they are not needed, which blows up the required memory for the decomposition when computing U,S,V. This leads the the memory allocation spikes which I show in the original question.

Possible solutions

I see two possible solutions for this problem which can be implemented in TensorLy.

1. Switch the pytorch svd backend from torch.svd (deprecated) to torch.linalg.svd and expose the driver argument in the tensorly svd and decomposition API. By default this archives two things: a) tensorly moves to the preferred pytorch svd interface, and b) the user can specify the cuSolver driver for the SVD depending on their needs. By default this keeps the tensorly svd with the PyTorch backend like it was before, however, by dispatching to the gesvda we can compute the approximate SVD with a different solver. The gesvda should only be used under some specific conditions, but does not allocate as much memory as the current interface.
2. Add torch.svd_lowrank option as an svd interface for the pytorch backend which implements the randomized_svd (Halko et al. 2009) directly and allocates less memory.

In the figure below I show the difference between the memory allocation for the current svd computation (left), the torch.linalg.svd using the approximate SVD gesvda driver (middle), and the torch.svd_lowrank (right). All SVD calculations use the randomized SVD algorithm and factorize the same data tensor using the TT decomposition.

In this case, the solutions I listed above decrease the required memory to decompose the tensor by a factor of ~2 (!), or in absolute terms by 4-6 Gb, adjusting for base load.

I would be interested in putting together a PR to migrate TensorLy (on PyTorch backend) to the torch.linalg.svd with a driver selection, or adding the torch.svd_lowrank to the randomized_svd/svd_interface for the PyTorch backend, or both. Would this be a welcome contribution @JeanKossaifi? Since an SVD interface change for the PyTorch backend might impact a lot of the package, I would like a maintainers opinion on this before I start.

[^1] Note: torch.svd is being deprecated in favor of torch.linalg.svd
[^2] Note that it might also dispatch it to gesvd via cusolverDnGesvd which is being deprecated in favor of cusolverDnXgesvd in the next major CUDA/ which apparently will have better (pre) memory allocation. The driver in tensorly.svd cannot be set, however, in torch.linalg.svd it is settable via the driver argument.
[^3] gesvdj has an economy mode that allocates memory more economically, however, I have not tested that yet, and I think that might be a question for PyTorch rather than TensorLy anyway.