A lightweight, high-performance deep learning inference framework built in Rust. Zen-Infer provides a clean, modular architecture for deploying neural networks in production environments with minimal dependencies and maximum performance.

• Lightweight: Minimal dependencies, small binary size
• High Performance: Optimized for CPU inference with potential for GPU acceleration
• Modular Design: Pluggable operator system for easy extension
• Memory Safe: Rust's ownership system prevents memory leaks and data races
• Cross Platform: Support for multiple architectures and operating systems
• Easy Integration: Simple API for loading and running PyTorch models

• Installation
• Quick Start
• Architecture
• API Reference
• Examples
• Contributing
• License

• Rust 1.70 or later
• Python 3.8+ (for model training and export)

# Clone the repository
git clone https://github.com/yyun543/zen-infer.git
cd zen-infer

# Build the Rust library
cd rust/zen_infer
cargo build --release

# Run tests
cargo test

# python/simple_net_train.py
import torch
import torch.nn as nn

class SimpleNet(nn.Module):
    def __init__(self):
        super(SimpleNet, self).__init__()
        self.linear1 = nn.Linear(2, 4)
        self.relu = nn.ReLU()
        self.linear2 = nn.Linear(4, 1)
        self.sigmoid = nn.Sigmoid()
    
    def forward(self, x):
        x = self.linear1(x)
        x = self.relu(x)
        x = self.linear2(x)
        x = self.sigmoid(x)
        return x

# Train and export model
model = SimpleNet()
# ... training code ...
torch.save(model.state_dict(), 'simple_net.pt')

# Export weights and model structure
python export_weights.py
python export_model_structure.py

use zen_infer::{Executor, Tensor, Shape};
use std::collections::HashMap;

fn main() -> Result<(), String> {
    // Load model weights and structure
    let weights = load_weights_from_json("weights.json")?;
    let graph = load_model_structure("model_structure.json", weights)?;
    
    // Create executor
    let executor = Executor::new();
    
    // Prepare input data
    let mut inputs = HashMap::new();
    inputs.insert(
        "input".to_string(),
        Tensor::new(vec![1.5, 2.0], Shape::new(vec![1, 2]))?
    );
    
    // Run inference
    let outputs = executor.execute(&graph, inputs)?;
    
    // Get results
    if let Some(result) = outputs.get("output") {
        println!("Prediction: {:.4}", result.data()[0]);
    }
    
    Ok(())
}

Zen-Infer follows a modular, layered architecture:

┌─────────────────────────────────────────┐
│              API Layer                  │
├─────────────────────────────────────────┤
│            Executor                     │
├─────────────────────────────────────────┤
│            Graph Engine                 │
├─────────────────────────────────────────┤
│         Operator Registry               │
├─────────────────────────────────────────┤
│         Tensor Operations               │
└─────────────────────────────────────────┘

• Tensor: Multi-dimensional array with optimized memory layout
• Operators: Pluggable computation units (Linear, ReLU, Sigmoid, etc.)
• Graph: Computational graph representation
• Executor: Inference engine with topological sorting
• Registry: Operator registration and discovery system

// Create tensors
let tensor = Tensor::new(vec![1.0, 2.0, 3.0], Shape::new(vec![1, 3]))?;

// Matrix multiplication
let result = tensor1.matmul(&tensor2)?;

// Element-wise operations
let relu_output = tensor.relu();
let sigmoid_output = tensor.sigmoid();

// Build computational graph
let mut graph = Graph::new();
graph.add_node(Node {
    id: "linear1".to_string(),
    op_type: "Linear".to_string(),
    inputs: vec!["input".to_string()],
    outputs: vec!["linear1_out".to_string()],
    params: linear_params,
});

// Register custom operators
registry.register("CustomOp".to_string(), Arc::new(CustomOperator));

This example demonstrates a complete workflow from training to inference:

1. Training (python/simple_net_train.py): Train a neural network for customer churn prediction
2. Export (python/export_*.py): Export PyTorch model to Zen-Infer format
3. Inference (rust/zen_infer/src/lib.rs): Run inference with Rust

Currently supports:

• ✅ Simple feedforward networks
• ✅ Linear layers with ReLU/Sigmoid activations
• 🔄 Convolutional networks (planned)
• 🔄 Recurrent networks (planned)
• 🔄 Transformer models (planned)

We welcome contributions! Please see our Contributing Guide for details.

# Clone and setup
git clone https://github.com/yyun543/zen-infer.git
cd zen-infer

# Install development dependencies
rustup component add rustfmt clippy

# Run tests
cargo test

# Run linter
cargo clippy

# Format code
cargo fmt

1. Implement the Operator trait
2. Register in the Registry
3. Add tests
4. Update documentation

Example:

pub struct CustomOperator;

impl Operator for CustomOperator {
    fn name(&self) -> &str { "CustomOp" }
    
    fn execute(&self, inputs: &[Tensor], params: &HashMap<String, Tensor>) 
        -> Result<Vec<Tensor>, String> {
        // Implementation
    }
}

This project is licensed under the MIT License - see the LICENSE file for details.

• Inspired by PyTorch's design principles
• Built with Rust's excellent ecosystem
• Thanks to all contributors and the open source community

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Email: yyun543@gmail.com

Zen-Infer: Making deep learning inference simple, fast, and reliable. 🚀