torchdbg is two things:

• A tracer that dumps all PyTorch operations that occur in your program to a structured log (formatted compatibly with tlparse)

• A React UI for visualizing the traces produced above, in the same style as a single-stepping debugger.

This is an Easter vacation hack from @ezyang, I don't currently have plans to keep pushing this further but maybe the community is willing to pick this up and run with it.

You need a reasonably recent PyTorch nightly (it needs pytorch/pytorch#120289) for this to work.

To use the Python tracer, pip install torchdbg and then in your code:

import torchdbg

with torchdbg.LoggingMode():
    ...  # your code here

By default, we do not log anything: run with TORCH_TRACE=/tmp/trace to specify the log directory. If you are running a distributed training run, multiple trace files will be generated per rank.

A live version of the UI is here: https://ezyang.github.io/torchdbg/ It is entirely client side, the trace files never leave your browser. Use the slider at the bottom to advance forward/back in the trace.

To run it locally, first, run the development server:

npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev

Open http://localhost:3000 with your browser to see the result.

Upload one of the generated trace files to the UI.