XPU-RT is an adaptable full-stack end-to-end (E2E) compilation and scheduling flow for efficient mapping of robotic multi-model workloads onto heterogeneous shared-memory SoCs.

This project is under active development. If you would love to contribute or if you find any issues, please do so by opening a pull request or filing an issue on GitHub.

git clone https://github.com/ucb-bar/XPU-RT.git
cd XPU-RT
git submodule update --init --recursive

Merlin provides the compiler toolchain, IREE runtime, and cross-compilation support used by XPU-RT. It ships as a git submodule under merlin/.

• Conda (Miniconda or Mamba)
• Internet access for initial setup (toolchain downloads, submodule clones)

conda env create -f merlin/env_linux.yml
conda activate merlin-dev
uv sync

The one-step setup script handles everything (toolchain + host compiler + target runtime):

bash setup.sh

Or run merlin commands individually (from within the merlin/ directory):

cd merlin

# Install SpacemiT cross-compilation toolchain
uv run tools/merlin.py setup toolchain --toolchain-target spacemit

# Build host compiler tools
uv run tools/merlin.py build --profile vanilla --config release

# Build SpacemiT target runtime (includes xpu-rt plugin)
uv run tools/merlin.py build --profile spacemit --config perf

cd ..

See merlin/docs/getting_started.md and merlin/docs/reference/cli.md for the full Merlin CLI reference.

After building merlin, run these from the XPU-RT root:

runtime/scripts/compile_all_models.sh   # compile VMFB files for all models
runtime/scripts/profile_remote.sh       # profile on target device (e.g. BananaPi)

For runtime-specific setup and usage details, see runtime/README.md.

During active merlin development, you can use a standalone merlin checkout instead of the submodule. Either symlink it:

rm -rf merlin && ln -s /path/to/your/merlin merlin

Or set the MERLIN_DIR environment variable (respected by setup.sh, compile_all_models.sh, and build_runtime.sh):

export MERLIN_DIR=/path/to/your/merlin
bash setup.sh

Create data/toplevel/networks_periodic_profile.json if there is none, and add entries like:

"mlp": {
  "id": 1,
  "identifier":            # model name
  "dispatch_deps_path":    # path to model json 
  "period":                # Duration in millisec between excution windows (inverse of frequency)
  "window_duration":       # Duration in millisec for model to finish after window start 
}

Run basic demos on top-level network graph:

python scripts/run_xpurt_schedule.py --profiled

The optimal schedule of your workloads on your target will be found in schedules/scheduled_networks_periodic_profiled.json with visualization in 'plots/iree_combined_schedule_period.png' after it finishes.

Run greedy scheduler variant:

python scripts/run_greedy_schedule.py --use-grouped

XPU-RT/
├── xpu-rt/                    # Python scheduler core modules
│   ├── scheduler.py
│   ├── workload.py
│   ├── workload_factory.py
│   ├── packing.py
│   ├── plot.py
│   ├── schedule_validation.py
│   └── pytorch_workload/      # Sample model artifacts + dispatch JSON inputs
├── scripts/                   # Python entry points for experiments/scheduling
├── runtime/                   # Scripts for compile/profile flow + optional custom tools
│   ├── scripts/*.sh           #   compile_all_models, profile_remote, etc.
│   └── tools/                 #   Custom tool sources (links merlin's xpu-rt archive)
├── data/                      # Collected benchmark/profile/scheduling outputs
├── merlin/                    # Git submodule (compiler/runtime/tooling upstream)
│   ├── tools/merlin.py        #   Unified CLI: build, compile, setup, benchmark, ...
│   ├── samples/common/xpu-rt/ #   XPU-RT runtime library (baseline + scheduler runners)
│   ├── samples/SpacemiTX60/   #   SpacemiT-specific sample binaries
│   └── models/                #   Model definitions (MLIR/ONNX sources)
├── env.yml                    # Conda environment
└── setup.py                   # Editable pip install config

1. runtime/scripts/compile_all_models.sh -> calls merlin/tools/merlin.py compile
2. runtime/scripts/compile_all_models.sh -> compiles models under merlin/models/...
3. Pre-built runner binaries come from merlin/build/<profile>/runtime/plugins/merlin-samples/:
   • merlin-baseline-async — baseline topo-order dispatch runner
   • merlin-dispatch-scheduler — two-cluster scheduled dispatch runner
4. XPU-RT runtime C API headers: merlin/samples/common/xpu-rt/*.h
5. Standalone archive for custom tools / Zephyr: merlin/build/<build-name>/runtime/src/iree/runtime/libxpurt_standalone.a

1. runtime/scripts/compile_all_models.sh generates VMFB + graph artifacts into gen/vmfb/... (using Merlin compiler).
2. runtime/scripts/profile_remote.sh runs topology benchmarks remotely and writes CSV results to gen/profile/....
3. scripts/run_xpurt_schedule.py reads profiled CSVs from gen/profile/... and combines them with dispatch graph JSON inputs to produce schedules.
4. Final scheduling outputs and logs are stored under data/... and script output directories.

1. The Python scheduler modules are sourced from xpu-rt/*.py and installed via setup.py.
2. Runtime C tooling in runtime/ is separate from Python scheduling code and is focused on Merlin/IREE integration.
3. If submodule contents are missing, runtime build/profile scripts will fail early.