Repro: patch in #139213 (needed for an error to show up), then run the following script:

import torch
import torch._functorch.config

@torch.library.custom_op("export::foo", mutates_args={}) # E: Untyped decorator makes fun
def foo(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor:
    return x * y

class Foo(torch.nn.Module):
    def forward(self, x, y): # E: Function is missing a type annotation  [no-untyped-def]
        return foo(x, y)

model = Foo()
inputs = (torch.randn(1, 3), torch.randn(2, 1))
with torch._functorch.config.patch(fake_tensor_propagate_real_tensors=True):
    ep = torch.export.export_for_training(model, inputs)
    nodes = list(ep.module().graph.nodes)
    ep.run_decompositions({})

What's going on is that:

• the included PR allows real_tensor_prop to infer a meta kernel for an operator during export
• the exported program does not include real tensors along with the FakeTensors (should it?)
• run_decompositions does some re-tracing without propagate_real_tensors because there are no real tensors
• run_decompositions errors due to no meta (the inferred meta kernel isn't persistent)

A fix could be that "if you're not decomposing the operator, then run_decompositions uses the existing FakeTensors to infer a meta kernel". But there's a more general question of if there are any more downstream graph passes that will have problems due to not having access to the real tensors.

cc @ezyang @chauhang @penguinwu @avikchaudhuri @gmagogsfm @zhxchen17 @tugsbayasgalan @angelayi @suo @ydwu4