Skip to content
XimikPublic
forked from ranvier-labs/tyr

About

A dependently-typed machine learning framework for Lean 4, providing compile-time tensor shape verification.

Resources

Stars

0 stars

Watchers

0 watching

Forks

 
 

Repository files navigation

Tyr

A deep learning framework for Lean 4 with a typed tensor facade over LibTorch.

Overview

Use torch.Tensor for shape, dtype, and device indices in model code. Native adapters use RawTensor handles internally; checked constructors validate external metadata before exposing indexed tensors.

import Tyr.Typed
open torch

def forward : Tensor { shape := #[32, 512], dtype := .Float32, device := .CPU } :=
  let x := Tensor.ones #[32, 768]
  let w := Tensor.ones #[512, 768]
  x.linear w

-- Replacing w with Tensor.ones #[768, 512] is a type error.
-- Raw handles cross into this facade through Tensor.ofTensor, which checks
-- their actual shape and dtype. Native results enter through Tensor.ofRaw!.

Dependencies

Lean 4

Install elan (the Lean version manager):

curl https://raw.githubusercontent.com/leanprover/elan/master/elan-init.sh -sSf | sh

Open a new shell (or run source ~/.elan/env) so lake is on PATH. The correct Lean nightly is pinned in lean-toolchain and will be installed automatically on first lake build.

C++20 Compiler

  • macOS: Xcode command line tools (xcode-select --install)
  • Linux: GCC 10+ (sudo apt install build-essential)

Native dependencies

Third-party libraries and sources are pinned in deps/ and fetched into external/. This needs git, curl and unzip. Pick the CPU or CUDA build:

deps/fetch.sh cpu     # or: deps/fetch.sh cuda

Quick Start

Building

# Configure once (Lake keeps it until the next `lake -R`); must match deps/fetch.sh:
lake -R build test_runner                             # CPU
# lake -R -Kcuda=/usr/local/cuda build test_runner    # CUDA, with this toolkit

# Build the test runner (a good first build)
lake build test_runner

# Build specific executables
lake build TrainGPT
lake build TrainDiffusion
lake build TrainNanoChat
lake build FluxDemo

# Build everything (slow: every example and benchmark executable)
lake build

Running

lake exe builds an executable if needed and runs it; arguments follow its name:

lake exe TrainGPT

Running Tests

lake test                            # builds and runs test_runner
lake exe test_runner --filter GPT    # with arguments

Environment Variables

All optional.

Dependencies (deps/fetch.sh):

Variable Effect
TYR_DEPS_CACHE download cache directory (default external/.cache)

Build configuration (Lake options, given together on lake -R; Lake keeps them until the next lake -R, which replaces all of them):

Option Effect
-Kcuda=<toolkit> CUDA build with this toolkit, which must contain bin/nvcc. Without it: CPU build (CUDA kernels replaced by stubs). Checked against the libtorch from deps/fetch.sh
-Kgpu=<name> with -Kcuda, required: GPU to build kernels for, H100, A100, B200, B300 or GB10

Build:

Variable Effect
TYR_MAKE_JOBS parallel jobs for the native make build (default: CPU count)

See docs/ffi-and-build.md for finer GPU and compiler overrides.

Runtime:

Variable Effect
TYR_DEVICE cpu, cuda, mps or auto; device used by the examples and model loaders
TYR_VERBOSE_ERRORS=1 print the full libtorch report when a libtorch error crashes the program
TYR_DEBUG_MPS=1 print MPS (Apple GPU) availability diagnostics

Documentation

Per-component guides live in docs/ — covering the core tensor/FFI layer, training stack, model families, scientific computing, and the C++/build internals. An exhaustive API reference can be generated with doc-gen4 (see docbuild/).

Examples

See Examples/README.md for detailed per-example documentation.

Example Description Build target
TrainGPT Character-level GPT on Shakespeare lake build TrainGPT
TrainDiffusion Discrete masked diffusion on ASCII text lake build TrainDiffusion
TrainNanoChat Modded-nanogpt distributed training lake build TrainNanoChat
FluxDemo Flux Klein 4B image generation lake build FluxDemo
NanoProof Transformer theorem prover (model only) Part of Examples lib

Distributed NanoChat (GPU Node)

Use the helper scripts to run TrainNanoChat under torchrun without pulling in a mismatched CUDA module stack:

# default: debug smoke run on 2 GPUs
./scripts/nanochat/run_train_torchrun.sh

# explicit 4-GPU run
NPROC_PER_NODE=4 ./scripts/nanochat/run_train_torchrun.sh \
  --debug --iterations 2 --data data/nanochat --val data/nanochat

# scaling check (1/2/4 GPUs by default)
./scripts/nanochat/bench_distributed.sh

Notes:

  • Override process counts in the benchmark script with SIZES="2 4" (or any space-separated list).

Key Concepts

Typed and Raw Tensors

Tensor σ tracks shape, dtype, and a concrete device in one static specification:

def project {m k n : Nat}
    (x : Tensor { shape := #[m, k], dtype := .Float32, device := .CPU }) (w : Tensor { shape := #[k, n], dtype := .Float32, device := .CPU }) :
    Tensor { shape := #[m, n], dtype := .Float32, device := .CPU } :=
  x.mm w

The native carrier is RawTensor; model code uses indexed Tensor σ values. Native results enter through Tensor.ofRaw!, which aborts if a result contradicts its type; external data is checked on its own metadata where it enters. Tensor.reshape requires equal element counts and treats #[] as a scalar. Runtime-dependent shapes can be packaged existentially without losing the tensor's shape index.

TensorStruct Typeclass

Generic traversal over structures containing tensors:

class TensorStruct (α : Type) where
  map : (∀ {σ}, Tensor σ → Tensor σ) → α → α
  mapM {m : Type → Type} [Monad m] : (∀ {σ}, Tensor σ → m (Tensor σ)) → α → m α
  zipWith : (∀ {σ}, Tensor σ → Tensor σ → Tensor σ) → α → α → α
  fold {β : Type} : (∀ {σ}, Tensor σ → β → β) → β → α → β

Use Vector n α instead of Array α for type-safe zipWith operations.

Data Loading

Batch dimensions may come from runtime configuration while remaining in the tensor type:

let iter := BatchIterator.new shard 8 256
let (batch, iter') ← iter.next
-- batch : Option (Tensor { shape := #[iter.batchSize, iter.seqLen], dtype := .Int64, device := .CPU })

SafeTensors Type Provider

Tyr includes a Lean command-level type provider for SafeTensors schemas:

import Tyr.SafeTensors

open torch

-- Works with a single .safetensors file or a sharded directory.
-- If `model.safetensors.index.json` exists, introspection follows `weight_map`.
safetensors_type_provider "/path/to/model_dir_or_file" as ModelWeights

def inspectWeights : IO Unit := do
  IO.println s!"discovered tensors: {ModelWeights.tensorCount}"

  -- Per-tensor typed loader + typed schema metadata
  let tokEmbed ← ModelWeights.load_model_embed_tokens_weight
  IO.println s!"embed dtype: {ModelWeights.model_embed_tokens_weightSpec.dtype}"
  IO.println s!"embed shape: {tokEmbed.runtimeShape}"

  -- Hierarchical aggregate generated from tensor names
  let weights ← ModelWeights.loadAll
  let qProj := weights.model.layers[0]!.self_attn.q_proj.weight
  IO.println s!"q_proj shape: {qProj.runtimeShape}"

  -- Hierarchical subtree loaders are also generated
  let decoder ← ModelWeights.model.load
  IO.println s!"loaded subtree"

Notes:

  • Tensor schema dtype metadata uses the core torch.DType type (not raw strings).
  • For sharded checkpoints, unsafe index shard paths (absolute paths or .. traversal) are rejected.

FFI Reference Counting

The C++ bindings use careful reference counting. See cc/src/tyr.cpp header for details:

  • borrowTensor(): Shared ownership, auto-cleanup
  • giveTensor(): Transfer ownership to Lean
  • lean_dec(): Required after extracting from lean_obj_arg, not for b_lean_obj_arg

Monitor tensor leaks via get_live_tensors which tracks outstanding C++ tensors.

Development

Adding New Tensor Operations

  1. Add Lean declaration in Tyr/RawTensor.lean or a Tyr/RawTensor/ submodule with @[extern "lean_torch_xxx"]
  2. Implement in cc/src/tyr.cpp following reference counting conventions
  3. Rebuild: lake build

Project Structure

  • lakefile.lean - Lake build configuration
  • lean-toolchain - Lean version specification
  • cc/ - C++ FFI bindings (LibTorch wrapper)
  • Tyr/ - Core framework (tensors, modules, optimizers, distributed)
  • Examples/ - Training scripts and model implementations
  • Tests/ - Test suites

Commit Template

This repo uses scoped conventional commit subjects:

type(scope): summary

A commit message template is included at .gitmessage. Enable it and the hooks locally:

git config commit.template .gitmessage
git config core.hooksPath .githooks

Included hooks:

  • pre-commit: fails on staged whitespace errors and conflict markers
  • commit-msg: enforces type(scope): summary (e.g. feat(qwen35): add video stream patchify)
  • pre-push: validates pushed commit subjects with .githooks/check-commit-message.sh

CI also enforces this format on pull requests, for both commit subjects and the PR title (a squash merge uses it as the commit subject), using the same checker.

Manual check example:

./.githooks/check-commit-message.sh "feat(qwen35): add video stream patchify"

License

Licensed under the Apache License, Version 2.0.

About

A dependently-typed machine learning framework for Lean 4, providing compile-time tensor shape verification.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages