LM7

Keep your PyTorch model. Change the hardware target, not your application.

LM7 is a PyTorch-first compiler orchestration layer for local inference. It keeps the normal nn.Module interface while detecting hardware, selecting an available compiler, moving inputs, caching compiled variants, and handling controlled fallback.

import lm7

compiled = lm7.compile(model.eval(), target="auto")
output = compiled(example_input)

On CPU, NVIDIA, AMD, Intel GPU, and Apple Silicon, LM7 often does use torch.compile with TorchInductor underneath. LM7 is not another compiler and does not replace Inductor, TensorRT, OpenXLA, OpenVINO, or the other toolchains it integrates.

torch.compile(model) is a good answer when you already know the hardware and compiler you want to use. LM7 is for the case where the hardware can change.

The missing layer is everything around the compiler call:

• detecting NVIDIA, AMD ROCm, Intel XPU, Apple MPS, TPU, and other accelerators;
• normalizing their different device semantics;
• selecting an available compiler for the resolved target;
• moving nested inputs and caching variants by input signature;
• handling first-call compilation failures and controlled fallback;
• explaining backend selection and managing artifacts across compiler stacks.

Some targets are not an Inductor call at all: TPU uses PyTorch/XLA and OpenXLA, Intel NPU uses OpenVINO, and other accelerators bring their own compiler and runtime.

lm7.compile(model, target="auto")
lm7.compile(model, target="nvidia", backend="tensorrt")
lm7.compile(model, target="amd")
lm7.compile(model, target="tpu")
lm7.compile(model, target="intel:npu")

PyTorch and hardware vendors provide the compilers. LM7 provides the vendor-neutral orchestration layer between them.

If one torch.compile(model) call already covers your machine and deployment needs, you probably do not need LM7. If your PyTorch application needs to survive a change of hardware, LM7 is intended to make that change boring.

See what LM7 replaces for the concrete per-vendor code and behavior it centralizes.

Use LM7 when your PyTorch application needs to run on hardware you do not want to hard-code: a developer laptop today, NVIDIA tomorrow, a TPU or another accelerator later.

LM7 is useful when you want to:

• ship the same PyTorch application to machines with different hardware;
• compare compiler backends without rewriting application code;
• detect hardware and compiler availability at runtime;
• build, inspect, and load artifacts through one interface.

If you have one fixed target and already know the vendor stack you want, use that stack directly. A fixed NVIDIA deployment running vLLM, for example, does not need LM7.

LM7 sits between one PyTorch model and the vendor toolchains that compile it. lm7.compile() returns a normal callable; lm7.export() writes a versioned .lm7 artifact. Both accept the same target vocabulary.

• Targets and backends are separate. A target says where the model runs (cpu, nvidia, apple, tpu); a backend says which compiler gets it there (inductor, tensorrt, openxla).
• Detection is automatic. target="auto" prefers a detected accelerator and otherwise uses CPU.
• Compilation is lazy and signature-aware. The first call compiles; compiled variants are cached by input signature.
• Fallback is controlled. A failed backend can warn and fall back to eager, or fallback="error" can stop immediately.
• Selection is inspectable. lm7 explain --target auto reports which backend would be selected and why.

These machines have executed LM7 paths on physical hardware. Only CPU and Apple MPS run in CI; the remaining machines were exercised manually. Exact parts, backends, workloads, and known gaps are recorded in tested hardware.

Integration means that LM7 has target and backend code for a toolchain. It does not mean every row has run on physical hardware.

AMD ROCm GPU, Intel XPU, Tenstorrent, Intel NPU, and Trainium have not run through LM7 on real hardware. See tested hardware and limitations for the evidence behind each row.

LM7 requires Python 3.10+ and a PyTorch build matching the target machine. It does not install GPU drivers, CUDA or ROCm, Xcode, PyTorch/XLA, or vendor toolchains.

git clone https://github.com/lmontigny/lm7.git
cd lm7
uv venv --python 3.12
uv pip install -e .
uv pip install torch --torch-backend=auto

You still install the driver and compiler/runtime required by your hardware. LM7 removes the per-vendor application glue and tells you what is missing through lm7 doctor.

lm7 doctor
lm7 targets
lm7 backends
lm7 explain --target auto

Then compile a model without hard-coding its device:

compiled = lm7.compile(model.eval(), target="auto")
result = compiled(example_input)

print(compiled.target, compiled.selected_backend)

Per-hardware setup: CPU · NVIDIA · AMD ROCm · Apple Silicon · Google TPU · Tenstorrent.

With backend="auto", LM7 chooses the highest-priority installed backend for the resolved target. Export-only integrations are never selected by lm7.compile. See JIT vs. AOT, the architecture guide, and the documentation index for details and optional dependencies.

Run or generate with a Hugging Face model:

uv pip install -e ".[hf]"
lm7 model run hf://HuggingFaceTB/SmolLM2-135M-Instruct \
  --prompt "The capital of France is" --target auto
lm7 model generate hf://HuggingFaceTB/SmolLM2-135M-Instruct \
  --prompt "The capital of France is" --max-new-tokens 32 --target nvidia

See model compatibility and compiled generation.

Serve a model for local validation:

uv pip install -e ".[serve,hf]"
lm7 model serve hf://HuggingFaceTB/SmolLM2-135M-Instruct --target auto

This is a single-user validation server, not a production serving engine. For production NVIDIA serving, --backend vllm hands execution to vLLM. See serving.

Export and reload a versioned artifact:

lm7.export(model, args=(example_input,), target="cpu", output="model.lm7")
loaded = lm7.load_artifact("model.lm7")
output = loaded(example_input)

Choose an export backend for AOTInductor, TensorRT, OpenVINO, ONNX Runtime, IREE, ExecuTorch, QNN, Core ML, or StableHLO payloads. See JIT vs. AOT and artifact inspection.

Quantize a validated Hugging Face model:

uv pip install -e ".[hf,torchao]"
lm7 model run hf://HuggingFaceTB/SmolLM2-135M-Instruct \
  --target cpu --quantize int8

Quantization is admitted per model and mode rather than promised universally. See quantization for measurements and accuracy checks.

Examples and reproducible benchmark harnesses live in examples/ and benchmarks/.

Projects such as ZML and Roofline.ai pursue the same broader goal of making ML workloads portable across heterogeneous hardware.

LM7 makes a narrower architectural bet: hardware portability does not necessarily require another compiler or runtime. Existing stacks such as TorchInductor, TensorRT, OpenVINO, OpenXLA, and ExecuTorch already contain substantial hardware-specific work. LM7 provides a neutral layer above them instead of replacing them.

Start with:

• Limitations — what LM7 does not do, per backend.
• Tested hardware — physical machines and exact gaps.
• Architecture — targets, backends, planner, artifacts.
• What LM7 replaces — per-vendor application glue.
• JIT vs. AOT — compilation timing and artifact rules.
• Development and testing — running the suite and GPU tests.

The documentation index links every hardware and backend guide.

Issues and pull requests are welcome. Run the checks before opening one:

uv pip install -e ".[dev]"
python -m pytest -q
python -m ruff check .
python -m ruff format --check .

Hardware-specific tests skip automatically when the device or toolchain is absent. See development and testing.

LM7 is licensed under the BSD 3-Clause License.