Pure Rust CUDA replacement -- cuBLAS, cuDNN, cuFFT, cuSPARSE, cuSOLVER, cuRAND and beyond in ~783K lines of safe Rust across 73 crates.

OxiCUDA replaces the entire NVIDIA CUDA Toolkit software stack with type-safe, memory-safe Rust code. The only runtime dependency is the NVIDIA driver (libcuda.so / nvcuda.dll); no CUDA SDK, no nvcc, no C/C++ toolchain is needed at build time. Optimized PTX assembly is generated directly from Rust data structures, and a built-in autotuner benchmarks kernel variants per GPU architecture to achieve near-peak throughput from Turing through Blackwell.

+---------------------------------------------------------------+
|   SciRS2  |  OxiONNX  |  TrustformeRS  |  ToRSh              |
|   (Scientific Computing / ML / Inference Ecosystem)           |
+-------------------------------+-------------------------------+
                                |
+-------------------------------v-------------------------------+
|                         OxiCUDA                               |
|                     (Pure Rust GPU)                            |
|                                                               |
|  Vol.1 Foundation (4 crates)                                  |
|  +----------+ +--------+ +---------+ +---------+             |
|  | Driver   | | Memory | | Launch  | | Runtime |             |
|  +----------+ +--------+ +---------+ +---------+             |
|                                                               |
|  Vol.2 Codegen (2 crates)                                     |
|  +-----------+ +------------+                                 |
|  | PTX Gen   | | Autotune   |                                 |
|  +-----------+ +------------+                                 |
|                                                               |
|  Vol.3 Linear Algebra    Vol.4 Deep Learning                  |
|  +-------------+         +-------------+                      |
|  | BLAS        |         | DNN         |                      |
|  +-------------+         +-------------+                      |
|                                                               |
|  Vol.5 Scientific Computing (4 crates)                        |
|  +------+ +--------+ +--------+ +------+                     |
|  | FFT  | | Sparse | | Solver | | Rand |                     |
|  +------+ +--------+ +--------+ +------+                     |
|                                                               |
|  Vol.6 Signal    Vol.7 Comp.Graph  Vol.8 Training (2)         |
|  +---------+     +----------+      +-------+ +-------+        |
|  | Signal  |     | Graph    |      | Train | | Quant |        |
|  +---------+     +----------+      +-------+ +-------+        |
|                                                               |
|  Vol.9 Inference (3 crates)        Vol.10 RL                  |
|  +-------+ +------------+ +----+   +------+                   |
|  | Infer | | Dist-Infer | | LM |   |  RL  |                   |
|  +-------+ +------------+ +----+   +------+                   |
|                                                               |
|  Backends (7 crates)                                          |
|  +----------+ +--------+ +-------+ +--------+                 |
|  | backend  | | prims  | | Metal | | Vulkan |                 |
|  +----------+ +--------+ +-------+ +--------+                 |
|  +--------+ +-------+ +-----------+                           |
|  | WebGPU | | ROCm  | | LevelZero |                           |
|  +--------+ +-------+ +-----------+                           |
+-------------------------------+-------------------------------+
                                |
+-------------------------------v-------------------------------+
|              libcuda.so  (NVIDIA Driver, runtime only)        |
|              No SDK  /  No nvcc  /  No C Toolchain            |
+---------------------------------------------------------------+

Vol.1 -- Foundation (4 crates, 26,438 SLoC)

• Dynamic driver loading via libloading -- zero build-time SDK dependency
• DeviceBuffer<T> with Rust ownership semantics -- Send + Sync, RAII
• Type-safe launch! macro with compile-time grid/block validation
• CUDA Runtime API layer for high-level device management

Vol.2 -- PTX Codegen & Autotuner (2 crates, 47,429 SLoC)

• Rust DSL that generates PTX IR covering SM 7.5 through SM 10.0
• Tensor Core support: WMMA, MMA, WGMMA instruction generation
• Built-in autotuner with 3-tier dispatch (cached / tuned / default)
• Disk-based PTX cache keyed by kernel hash + GPU architecture

Vol.3 -- BLAS (1 crate, 28,379 SLoC)

• Full BLAS Level 1/2/3 (axpy, gemv, gemm, trsm, syrk, ...)
• GEMM dispatch: SIMT, Tensor Core, Split-K paths
• Batched GEMM: standard, strided, grouped
• Precision coverage: F16, BF16, TF32, F32, F64, FP8
• Elementwise ops (relu, gelu, sigmoid, silu) and reductions (softmax, variance)

Vol.4 -- DNN (1 crate, 39,297 SLoC)

• Convolution: implicit GEMM, im2col, Winograd 3x3, direct, fused Conv+BN+Act
• FlashAttention forward/backward, PagedAttention, decode attention
• MoE: top-k routing, token permutation, fused MoE kernel
• Normalization: BatchNorm, LayerNorm, RMSNorm, GroupNorm
• Pooling: max, average, adaptive, global
• Resize: nearest, bilinear, bicubic
• Quantization: FP8, INT8, block-scaled FP4

Vol.5 -- Scientific Computing (4 crates, 62,511 SLoC)

• FFT: Stockham, radix-2/4/8, mixed-radix, Bluestein, C2C/R2C/C2R, 2D/3D
• Sparse: CSR/CSC/COO/BSR/ELL, SpMV, SpMM, SpGEMM, SDDMM, ILU(0)/IC(0)
• Solver: LU, QR, SVD, Cholesky, eigendecomp, CG, BiCGSTAB, GMRES
• Rand: Philox, MRG32k3a, XORWOW, Sobol, uniform/normal/Poisson

Vol.6 -- Signal Processing (1 crate, 12,276 SLoC)

• Audio: MFCC, STFT, Mel filterbank, spectral features
• Image: Gaussian blur, Sobel edge detection, morphological ops
• DCT: Types I-IV with fast algorithms
• DWT: Haar, Daubechies wavelets
• Filtering: IIR/FIR filters, Butterworth, Chebyshev
• Correlation: cross-correlation, autocorrelation

Vol.7 -- Computation Graph (1 crate, 6,563 SLoC)

• CUDA Graph capture API (StreamCapture, GraphCapture)
• Execution plan with dependency-sorted node scheduling
• Event-based inter-node synchronization
• Sequential + parallel graph executors

Vol.8 -- GPU Training (2 crates, 13,832 SLoC)

• Mixed precision training (AMP): FP16/BF16 + loss scaling
• Gradient accumulation and clipping; EMA (exponential moving average)
• LR schedulers: cosine, warmup, cyclic, polynomial
• GPU-fused optimizers: Adam, AdamW, SGD, RMSProp, LAMB
• Checkpointing (model save/load)
• Quantization: INT8/INT4/FP8 weight quantization, block-scaled

Vol.9 -- Inference Engine (3 crates, 17,909 SLoC)

• KV-cache with paged attention (PagedKvCache) and prefix caching
• Speculative decoding
• Distributed inference pipeline (tensor/pipeline parallelism)
• LM inference: BPE tokenizer, vocabulary management, sampling strategies

Vol.10 -- Reinforcement Learning (1 crate, 11,280 SLoC)

• Replay buffers: Uniform, Prioritized (PER), N-step
• Policy distributions: Categorical, Gaussian (SAC reparameterization), Deterministic
• Advantage estimators: GAE, TD(λ), V-trace, Retrace(λ)
• Loss functions: PPO, DQN, Double-DQN, SAC, TD3
• Observation/reward normalization with Welford running stats
• Environment abstractions: Env, VecEnv (auto-reset)

Backends (7 crates, 28,400 SLoC)

• Backend trait abstraction for multi-GPU-runtime portability
• CUB-equivalent GPU primitives (scan, reduce, sort, histogram)
• Metal (macOS), Vulkan Compute, WebGPU, AMD ROCm, Intel oneAPI (LevelZero)

OxiCUDA is built on a strict Pure Rust policy with minimal external dependencies. The entire codebase compiles with cargo build alone -- no C compiler, no Fortran runtime, no CUDA SDK, no nvcc, no pkg-config.

The only runtime requirement is the NVIDIA GPU driver (libcuda.so on Linux, nvcuda.dll on Windows). On macOS the crate compiles but returns UnsupportedPlatform at runtime.

use oxicuda::prelude::*;

fn main() -> Result<(), oxicuda::Error> {
    // Initialize driver and select GPU device
    let device = Device::get(0)?;
    let ctx = Context::new(device)?;
    let stream = Stream::new(&ctx)?;

    // Allocate device memory
    let mut d_a = DeviceBuffer::<f32>::zeroed(1024)?;
    let mut d_b = DeviceBuffer::<f32>::zeroed(1024)?;
    let mut d_c = DeviceBuffer::<f32>::zeroed(1024)?;

    // Copy host data to device
    d_a.copy_from_host(&host_a)?;
    d_b.copy_from_host(&host_b)?;

    // Launch a GEMM: C = alpha * A @ B + beta * C
    let handle = BlasHandle::new(&stream)?;
    handle.gemm(
        Transpose::None, Transpose::None,
        m, n, k,
        1.0f32,            // alpha
        &d_a, lda,
        &d_b, ldb,
        0.0f32,            // beta
        &mut d_c, ldc,
    )?;

    stream.synchronize()?;

    // Copy result back to host
    let mut result = vec![0.0f32; m * n];
    d_c.copy_to_host(&mut result)?;
    Ok(())
}

# Default build (no GPU features)
cargo build

# With all GPU features
cargo build --features "ptx,autotune,blas,dnn,fft,sparse,solver,rand"

# Full build (all features including backends)
cargo build --features full

# Check without GPU
cargo check --all-targets

# Unit tests (no GPU required)
cargo test

# Full test suite with GPU hardware
cargo test --features gpu-tests

# Run with nextest
cargo nextest run --all-features

Released (v0.2.0) -- 2026-06-16 (32,320 tests passing, ~1.06M SLoC, 73 crates)

• Vol.1: Driver, Memory, Launch, Runtime -- foundation layer (4 crates)
• Vol.2: PTX codegen DSL, autotuner engine (2 crates)
• Vol.3: Full BLAS L1/L2/L3 with Tensor Core GEMM, SYR2K two-operand cross-product variant
• Vol.4: Convolution, FlashAttention, MoE, normalization, pooling, quantization
• Vol.5: FFT, sparse, solver, RNG (4 crates)
• Vol.6: Signal processing -- audio/image DSP, DCT, DWT, IIR/FIR filters
• Vol.7: Computation graph -- capture API, dep-sorted scheduling, parallel executor
• Vol.8: GPU training -- AMP, optimizers, LR schedulers, checkpointing, quantization (2 crates)
• Vol.9: Inference engine -- KV-cache, speculative decode, distributed infer, LM (3 crates)
• Vol.10: Reinforcement learning -- replay buffers, policy dists, PPO/DQN/SAC/TD3
• Backends: Metal, Vulkan, WebGPU, ROCm, LevelZero (7 crates)
• Vol.17: Generative AI -- diffusion schedulers, CFG, VAE, LoRA
• Vol.18: Graph Neural Networks -- GCN/GAT/GraphSAGE/GIN, pooling
• Vol.19: State Space Models -- HiPPO-NPLR, S4D/S5, Mamba SSM, RWKV
• Vol.20: Vision Transformers & CLIP -- ViT, patch embedding, dual-tower CLIP
• Vol.21: Audio/Speech ML -- Conformer, Wav2Vec2, CTC/RNN-T, WaveNet, SpecAugment
• Vol.22: Time-Series Forecasting -- TCN, NHiTS, PatchTST, TimesNet, iTransformer, RevIN
• Vol.23: Bayesian Deep Learning -- variational inference, MC Dropout, Ensembles, Laplace
• Vol.24: Federated Learning -- FedAvg/FedProx/SCAFFOLD/FedAdam, DP, secure aggregation
• Vol.25: Neural Architecture Search -- DARTS, supernet, NSGA-II, hardware-aware predictor
• Vol.26--61: SSL, Adversarial, Multimodal, Continual, 3D Geometry, PINN, RLHF, Meta-Learning, NeRF, MoE, Tabular, Anomaly, Quantum, ANN, RecSys, Causal, PEFT, Distillation, OT, SNN, DP, HDC, Evolutionary, TDA, Tensor Networks, Sequence Models, PDE, Manifold, Statistics, Sketches, Survival, CVX, Compressed Sensing, Graph Algorithms, Numerical Analysis, 2D Geometry

Next

• Published documentation on docs.rs
• GPU hardware benchmark validation (CI regression tracking)
• v1.0 completion criteria verification (see TODO.md)

• API Documentation
• GitHub Repository
• COOLJAPAN Ecosystem
• Changelog

Licensed under the Apache License, Version 2.0. See LICENSE for details.

(C) 2026 COOLJAPAN OU (Team KitaSan)