Problem

In most combinatorial settings such as the ones we consider, the initial td (e.g. locs in a Euclidean routing problem) does not really change so we do not need to carry information about all the computational graph. This is why, unlike TorchRL, we modified the step() function of the environment here not to save all previous td (since they would just increase runtime).
However, this is not in general true when we consider dynamic / stochastic settings.

Solution

We should better explain why we do this and allow users to save intermediate states during decoding as an option, perhaps specifying the problem as static or dynamic instead of having the _torchrl_mode in here.

PS: optionally one could save the Tensordicts alongside actions here - i.e., saving each step inside of the DecodingStrategy from @LTluttmann upon request and giving back the full nested td as usually done in TorchRL

CC: @Furffico @cbhua