RustyTorch++ User Guide

Welcome to RustyTorch++, The Complete Rust AI/ML Platform - 62 crates, 300K+ lines of code, spanning from scientific computing to production LLM inference.

What is RustyTorch++?

RustyTorch++ is not just a deep learning framework - it's a complete AI/ML platform that provides:

LLM-First Architecture: Flash Attention (FP16 + FP8), Ring Attention for 16M+ tokens, speculative decoding with 6 strategies
GPU-Native Tensors: Zero-copy operations with CUDA, ROCm, Metal, WebGPU support
Production Ready: Continuous batching, serving API, WASM browser inference, monitoring
Memory Safe: Rust's ownership model prevents common GPU programming errors
Scientific Computing: CFD, FEA, Physics-Informed Neural Networks
Enterprise Features: Byzantine-robust federated learning, multi-tenant platform

RustyTorch++ vs Burn: Burn is a research framework with ~20 crates. RustyTorch++ is an enterprise platform with 62 crates, 2.8x faster LLM inference, and capabilities Burn doesn't have (Ring Attention, CFD, federated learning, production serving).

Framework Architecture

The framework is organized into 62 specialized crates across 8 categories:

Category	Crates	Key Features
Core	13	Tensor, runtime, autograd, Metal, bindings, SAE
Training	13	Flash Attention (FP8), distributed, RLHF, federated
Models	8	Vision, NLG, diffusion, multimodal, TTS
Production	7	Inference (speculative), serving, WASM+WebGPU, streaming
Specialized	11	CFD, FEA, PINNs, GNN, classical ML
Data	3	ETL pipelines, feature store, validation
Tooling	6	Benchmarks, evaluation, meta crates
Integration	1	Jupyter/Rustybooks

Quick Example

use rtx_tensor::{Tensor, Device};
use rtx_nn::{Linear, Module};

fn main() -> anyhow::Result<()> {
    // Create tensors on GPU
    let device = Device::cuda(0)?;
    let input = Tensor::randn([32, 784], &device)?;

    // Build a simple model
    let layer = Linear::new(784, 10, true, &device)?;
    let output = layer.forward(&input)?;

    println!("Output shape: {:?}", output.shape());
    Ok(())
}

Who Should Use This Guide?

This guide is designed for:

ML Engineers migrating from Python/PyTorch to Rust
Systems Engineers building high-performance inference systems
Researchers needing GPU-accelerated scientific computing
DevOps Engineers deploying ML models to production

Prerequisites

Rust 1.92+ (nightly recommended for Rust 2024 features)
CUDA 12.0+ for NVIDIA GPU support
Basic familiarity with machine learning concepts

Getting Help

Next Steps

Installation - Set up your environment
Quick Start - Run your first example
Architecture Overview - Understand the framework design

3.1 KiB Raw Blame History