Tiled Graph Inference Scheduler. An ahead-of-time compiler that tiles ML models to fit embedded devices with hard memory budgets.
Give it a model and a memory budget: ONNX, TFLite, or a PyTorch program saved with torch.export (.pt2). It partitions the compute graph into stages, tiles spatial operations, and emits a flat binary plan that the tigris-runtime executes with zero dynamic allocation.
If you already deploy TFLite Micro or ExecuTorch, tigris export rewrites the int8 .tflite file or the Cortex-M .pte program in row bands instead, so the runtime you ship runs it in less RAM with the same outputs.
On an embedded device with a few hundred KB of SRAM, most interesting models simply don't fit. The usual answer is to shrink the model: quantize harder, prune, pick a smaller architecture, and hope the accuracy hit is acceptable.
TiGrIS takes the other approach. It keeps the model you trained and rearranges the computation so that only a small working set lives in SRAM at any moment. Weights can stay in flash, and intermediate results that do not fit spill to slower writable memory such as PSRAM. What comes out is a binary plan that the runtime executes as a flat sequence of kernel calls, with no interpreter: activations get their addresses from caller-provided fast and slow arenas by bounded bump allocation, and inference never calls a general-purpose allocator.
pip install tigris-ml selects a wheel for the host. Native wheels include the
portable reference runtime for Linux x86-64 and ARM64 with glibc >= 2.28,
macOS x86-64 and ARM64, and Windows x86-64. The Linux wheels use
manylinux_2_28; older glibc, musllinux, Windows ARM, and 32-bit ARM Linux
install the pure Python wheel instead. A source distribution is also available.
The pure wheel supports inspection, compilation, analysis, code generation,
export of TFLite models and ExecuTorch programs, and zoo downloads. Running a
plan uses the bundled native library; without it run reports "Bundled host
runtime is unavailable". Running a TFLite model uses tflite-micro, and
running an ExecuTorch program builds a host runner once from an ExecuTorch
checkout, which fetches CMSIS-NN.
On platforms without a native wheel, build the host library from the matching
tigris-runtime release:
run cmake -S . -B build-host -DTIGRIS_BUILD_HOST=ON, then
cmake --build build-host --target tigris_host, and set TIGRIS_HOST_LIBRARY
to the resulting shared library file. tigris --version and tigris run --json
report the runtime's version and origin; an invalid override fails without falling back.
pip install tigris-ml
# Will this model fit in 256KB SRAM + 16MB flash?
tigris analyze mobilenetv2.onnx -m 256K -f 16Mwarning: input axis 0 (batch_size) is unset; using 1 (--input-shape overrides)
mobilenetv2.onnx float32, 65 operators
input input 1x3x224x224 float32
output output 1x1000 float32
fits 256.00 KiB fast memory, 58 stages, 53 tiled
memory
unscheduled 5.74 MiB
largest tensor 4.59 MiB 1x96x112x112
this plan 250.00 KiB 6.00 KiB headroom
slow memory 5.17 MiB
slow traffic 20.25 MiB per inference: 7.21 MiB written, 13.04 MiB read
smallest budget 37.50 KiB 37.47 KiB does not fit
flash
plan 13.31 MiB weights 13.30 MiB, overhead 8.32 KiB
as int8 3.33 MiB estimate
flash budget 16.00 MiB fits
Unscheduled, the model peaks at 5.74 MiB. TiGrIS schedules it into 256 KiB through temporal partitioning and spatial tiling. analyze runs on your laptop; no hardware required.
That model is a stock export with a free batch dimension. TiGrIS binds a dimension the model leaves open to 1 and says so; pass --input-shape input:4x3x224x224 to compile for a different one.
Three steps take a model from ONNX to a C file you can drop into your firmware project:
# 1. Analyze feasibility against a memory budget
tigris analyze model.onnx -m 256K -f 16M
# 2. Compile to a binary plan (weights read-in-place from flash)
tigris compile model.onnx -m 256K -f 16M --xip -o model.tgrs
# 3. Generate a backend-specific C harness for your target
tigris codegen model.tgrs --backend esp-nn -o model.cThe .tgrs plan is target-agnostic: the same file runs on an ESP32, a Cortex-M, or a POSIX host. The kernel backend is chosen at codegen time and decides which kernel library the generated C calls into. The operator and backend matrix shows which operators are native, which fall back, and which are rejected.
tigris compile writes a single .tgrs file holding the operator schedule, tile parameters, quantization tables, and the weights. With a host runtime, bundled or named by TIGRIS_HOST_LIBRARY, it first loads the plan with the runtime's loader and writes no plan the loader refuses. The pure wheel without TIGRIS_HOST_LIBRARY writes the plan without that check.
tigris codegen produces a C harness that loads the plan and hands it to the runtime: buffer and arena declarations, a target entry point, and the glue for reaching the plan bytes. --format app emits a standalone program. --format core emits a source and header for firmware that already owns its entry point, arenas, and input source, so several generated cores can coexist in one binary. The codegen reference documents the flags.
Link the harness against tigris-runtime and your kernel library, and you have a working inference binary.
The model zoo hosts precompiled plans with runtime requirements, input/output conventions, evaluation results, and per-model licenses. Public downloads need no account. List available builds before choosing a model:
tigris zoo list --category classification
tigris zoo list --runtime 0.9.1 --backend reference -m 256K
tigris zoo fetch MODEL -o downloaded-model
tigris inspect downloaded-model/model.tgrs
tigris codegen downloaded-model/model.tgrs --format core -o model.cFilters are optional. The newest published matching build wins. Runtime ranges
include both endpoints; a null maximum means no known upper compatibility bound.
Tested releases are reported separately. Matching a range does not claim that
every release in it has been tested. -m limits the fast arena, a second
-m limits the slow arena, and -f limits plan bytes. These are not total
application RAM or flash limits. Unspecified resources remain unconstrained.
The runtime for deployment on a device is supplied separately.
Use fetch --artifact ID to pin a build. Withdrawn builds are excluded from
automatic selection but remain explicitly retrievable with a warning. Downloads
verify file hashes and never replace an existing destination. manifest.json
preserves the original build metadata. download.json records the current
runtime constraints, tested releases, and pinned catalog revision; use it for
dependency integration. Catalog updates do not change artifact publication dates
or rebuild models. tigris zoo --offline ... uses the HF cache;
tigris zoo --catalog catalog.json ... reads a local zoo snapshot.
tigris inspect model.onnx
tigris inspect model.tgrs -v
tigris inspect model.tgrs --jsoninspect detects the format from file contents: ONNX, TFLite, an exported
PyTorch program (.pt2), an ExecuTorch program (.pte), or a compiled plan.
For a TFLite file it also states whether the model converts, and if not,
what stops it; for an ExecuTorch program it shows the interface and the RAM
the program records. It reads declared ONNX inputs,
outputs, operators, and initializers, or compiled plan interfaces, stages, tiling,
quantization, and memory records. -v adds details; --json emits complete
metadata with an inspection_version field and exact byte counts, without weight
values. Inspection is offline: it does not compile, execute, infer shapes, or load
external ONNX tensor data. Recorded plan memory is not measured runtime peak or
total application RAM. Plan schemas do not imply runtime version requirements.
Inspection does not validate kernel numerics. run executes portable kernels on
the host: TiGrIS's reference kernels for a plan, TFLite Micro's reference kernels
for a TFLite model, CMSIS-NN's portable C kernels for an ExecuTorch program.
ESP-NN, the DSP and MVE kernel paths and on-device latency run on the target.
Platform wheels include the reference C runtime. Run preprocessed tensors in
the stored axis order and declared interface dtype reported by inspect:
tigris run downloaded-model/model.tgrs --input downloaded-model/example-input.bin --output prediction.binCompare prediction.bin with example-output.bin numerically; the artifact's
readme.md names the reference (TFLite Micro or ONNX Runtime) and how close the
outputs must be.
tigris run model.tgrs --input input.npy --output prediction.npy
tigris --versionFor multiple inputs, repeat --input NAME=FILE. Use .npz for multiple named
outputs. Raw .bin files contain contiguous little-endian elements; .npy
inputs retain their declared shape and dtype. Output files must not exist.
--json reports execution metadata and arena peaks. These peaks exclude the
plan, executor workspace, and Python process memory. Host execution uses
only the portable float32 reference and int8 reference kernels. It checks plan
execution on that path, not ESP-NN or CMSIS-NN numerics or on-device latency.
It does not preprocess photos or CSV files.
import numpy as np
from tigris.runtime import Session
with Session("model.tgrs") as session:
outputs = session.run({"input": np.load("input.npy", allow_pickle=False)})
print(session.runtime_version, session.memory)Compiler and runtime releases share a version identifying the tested pair. Execution checks matching zoo dependency records when present and lets the C loader validate the plan. Schema versions, accepted schema ranges, and the host ABI retain their own compatibility meanings. Downloading and generating deployment code remain independent of the host runtime. See Contributing for native-library staging and wheel builds.
- Getting started: installation, first compile, deploying to ESP32
- What is TiGrIS: what the compiler does and where its result runs
- Core compatibility data: exact compiler/runtime releases and plan schemas
- Introducing TiGrIS: design, benchmarks, how tiling works
- CLI reference: every flag, every subcommand
TiGrIS is maintained by RAWS Labs. For applied embedded-ML engineering or collaboration, see raws.at.
git clone https://github.com/raws-labs/tigris
cd tigris
pip install -e . --group dev
pytest