rustytorch/CHANGELOG.md

# RustyTorch++ Changelog

## [December 28, 2025]

### Added - TIER 2 Feature Completeness

#### FlashAttention CPU/Metal Backward
- **Files**: `rtx-flash-attention/src/core.rs`, `src/lib.rs`
- CPU backward pass (`naive_attention_backward`)
- Metal backward pass (trait impl to kernel)
- 30 gradient tests passing

#### Nested vmap Support
- **Files**: `rtx-autograd/src/vmap.rs`
- `BatchDimStack` for tracking batch dimensions through nested calls
- `MultiBatchedVariable` for dimension collapsing/expansion
- `VmapLevelGuard` RAII guards for level tracking
- 12 vmap tests passing

#### Hessian-Vector Product (hvp)
- **Files**: `rtx-autograd/src/func.rs`
- `hvp(f, primals, tangents)` with forward-over-reverse mode
- `hvp_finite_diff` for numerical validation
- `vhp` (vector-Hessian product)
- Fixed f32 precision issues (use f64 intermediate, larger epsilon 1e-3)
- 7 hvp tests passing

#### Dynamic Shape Guards
- **Files**: `rtx-synthesis/src/aot_impl/shape_guards.rs`, `src/aot.rs`
- Shape guard generation (`generate_guards_from_operations`, `generate_symbolic_guards`)
- Recompilation triggers (`GuardCheckResult`, `GuardFailure`)
- Shape dimension types: Concrete, Symbolic, Bounded, Dynamic
- Shape bucketing for cache efficiency (`ShapeGuardManager`, `ShapeSignature`)
- 20 tests passing

#### Advanced Quantization (AWQ, GPTQ, SmoothQuant)
- **Files**: `rtx-compress/src/quantization/advanced.rs`
- AWQ (Activation-aware Weight Quantization) - per-channel scales, group quantization
- GPTQ (Accurate Post-Training Quantization) - Hessian-based, block-wise quantization
- SmoothQuant (migration difficulty from activations to weights) - configurable alpha
- QuantizedTensorData dequantization support
- 9 tests passing

---

### Added - TIER 3 Nice to Have Features

#### Distributed Checkpoint (DCP)
- **Files**: `rtx-distributed/src/dcp.rs`
- Async checkpointing (`AsyncSaveHandle` with progress tracking)
- Sharded state dict save/load (per-rank parallel I/O)
- Resumption from partial checkpoints (configurable `min_shards_for_partial`)
- World size change handling (shard redistribution)
- Atomic writes with fsync
- 11 tests passing

#### Context Parallel
- **Files**: `rtx-distributed/src/context_parallel.rs`
- Sequence dimension parallelism (`SequenceShardInfo` with even/uneven splits)
- Ring attention integration (`RingAttentionState`, `ring_attention` method)
- Long-context training support (max 128K tokens)
- KV cache distribution across CP ranks
- Async KV prefetch configuration
- 11 tests passing

#### Autograd Profiler
- **Files**: `rtx-autograd/src/profiler.rs`
- Operation timing (`ProfiledEvent` with duration tracking)
- Memory tracking per op (`MemorySnapshot`, allocation/deallocation)
- Gradient flow visualization (`GradientFlow`, DOT export)
- Bottleneck detection (`BottleneckInfo`, severity analysis)
- Chrome trace export for visualization
- RAII `RecordGuard` for scoped profiling
- 12 tests passing

#### SDPA Backend Auto-Selection
- **Files**: `rtx-flash-attention/src/backend_selector.rs`
- Automatic FlashAttention vs Math vs Memory-efficient selection
- Hardware detection for optimal backend (`HardwareCapabilities`)
- Fallback chain management (alternatives ranking)
- Performance-based scoring (sequence length, memory, speedup)
- Auto-tuning with performance history
- Debug mode and preferred backend options
- 12 tests passing

---

## [Unreleased] - December 20, 2025

### Added - Metal + WASM Benchmarking Suite (Phase 10)

Comprehensive benchmarking infrastructure for comparing RustyTorch++ against PyTorch MPS and WASM alternatives.

#### Metal Benchmarks (PyTorch MPS Comparison)
- **benchmarks/metal/bench_flash_attention.py**: Flash Attention benchmark with JSON output
  - BS1-BS256 scenarios, causal/non-causal modes
  - P50/P95/P99 latency statistics, throughput metrics
  - Tested: 0.2-12ms latency, 200M-3500M elements/sec on Apple Silicon

- **benchmarks/metal/bench_moe_mps.py**: Mixture of Experts benchmark
  - 4-16 expert configurations, top-k routing
  - Routing vs expert compute time breakdown
  - Tested: 1K-70K tokens/sec throughput

- **benchmarks/metal/bench_mamba_mps.py**: Mamba/SSM benchmark
  - Simplified selective scan for benchmarking
  - Long sequence tests (128-2048 tokens)
  - Tested: 28K-160K tokens/sec throughput

#### WASM Browser Benchmarks
- **benchmarks/wasm/comparison.html**: Visual browser benchmark page
  - Side-by-side comparison with ONNX.js and TensorFlow.js
  - Chart.js visualization of latency, throughput, memory
  - System capability detection (SIMD, threading, SharedArrayBuffer)

- **benchmarks/wasm/rtx_wasm_bench.js**: RustyTorch WASM benchmark module
- **benchmarks/wasm/package.json**: Node.js dependencies

#### Infrastructure
- **scripts/run_metal_benchmarks.sh**: Unified benchmark runner
  - `--quick` mode for fast validation
  - `--rust-only` / `--python-only` for selective runs
  - `--wasm` for browser benchmark setup
  - JSON output to benchmarks/reports/

#### Key Results (Apple Silicon M-series)
| Feature | PyTorch MPS | Expected RustyTorch | Target Speedup |
|---------|------------|---------------------|----------------|
| Flash Attention BS64 | 1.35ms | <0.7ms | 2x |
| MoE 8 experts | 99ms | <50ms | 2x |
| Mamba Seq1024 | 30ms | <15ms | 2x |
| WASM Inference | N/A | 3-5x vs ONNX.js | - |

---

### Added - WASM Inference Runtime (Phase 9)

WebAssembly inference runtime for deploying ML models in browsers and Node.js, implemented as new rtx-wasm-inference crate.

#### New Crate: rtx-wasm-inference
- **lib.rs**: Main WASM exports (~350 lines)
  - `WasmInferenceEngine` with async model loading and inference
  - `InferenceConfig` with fast/quality presets
  - `InferenceResult` with timing and token statistics
  - wasm-bindgen exports for JavaScript interop

- **runtime.rs**: Environment detection (~220 lines)
  - `WasmRuntimeInfo` detecting Browser, Node.js, Deno, Web Worker
  - SIMD, threading, and SharedArrayBuffer capability detection
  - Performance timing utilities

- **tensor.rs**: CPU tensor operations (~450 lines)
  - `WasmTensor` with matmul, softmax, activations (GELU, SiLU, ReLU)
  - Layer normalization and element-wise operations
  - `WasmKvCache` for transformer inference

- **model.rs**: Model loading (~360 lines)
  - `WasmModel` with forward pass and embedding lookup
  - `ModelConfig` with tiny/small presets
  - RMS normalization and FFN layers

- **tokenizer.rs**: Text processing (~200 lines)
  - `WasmTokenizer` with encode/decode
  - Special token handling (BOS, EOS, PAD, UNK)

- **quantization.rs**: Model compression (~250 lines)
  - INT8 and INT4 quantization
  - Per-block scaling with configurable block size
  - 4x compression ratio with INT8

#### Key Features
- Zero Python/CUDA dependencies - pure Rust compiled to WASM
- Browser and Node.js runtime detection
- Optional SIMD and threading support
- Memory-efficient quantized inference
- 17 tests passing

---

### Added - Continuous Batching System (Phase 7)

Production-ready continuous batching controller for LLM inference with iteration-level scheduling, implemented in rtx-serving-api crate.

#### New Files
- **continuous_batch.rs**: Full continuous batching implementation (~1000 lines)
  - `ContinuousBatchingController` for managing request lifecycle
  - `ContinuousBatchingConfig` with batch size, wait time, memory limits
  - `BatchRequest` with priority, SLA tracking, memory estimation
  - `ActiveBatch` with dynamic request joining/leaving
  - `Priority` enum: Low, Normal, High, Critical
  - `RequestState`: Queued, Processing, Generating, Preempted, Completed, Failed, Cancelled

#### Key Features
- Iteration-level scheduling: Requests join/leave batches at each decode step
- Preemption support: High-priority requests interrupt lower-priority batches
- Memory-aware batching: Respects KV-cache and GPU memory limits
- SLA enforcement: Deadline-driven scheduling with priority boosting
- Scheduling score: `priority_weight + urgency + wait_penalty + preemption_boost`
- 7 tests passing

---

### Added - Ring Attention for Long Context (Phase 8)

Ring attention implementation for 16M+ token context windows using distributed attention across device rings, implemented in rtx-transformers crate.

#### New Files
- **ring_attention.rs**: Ring attention with online softmax (~700 lines)
  - `RingAttention` for distributed attention computation
  - `RingAttentionConfig` with num_devices, chunk_size, overlap settings
  - `RingTopology` for device ring management and rotation
  - `SequenceChunk` for per-device sequence partitioning
  - `OnlineSoftmaxState` for numerically stable accumulation
  - `RotationBuffer` for efficient KV rotation
  - `RingAttentionBuilder` for fluent configuration

#### Key Features
- Sequence partitioning across device ring
- Online softmax for numerically stable accumulation
- Causal masking support for autoregressive models
- Overlapped communication with compute
- Gradient checkpointing support
- Memory estimation per device
- 10 tests passing

---

### Added - KV-Cache Optimization (Phase 6)

Entropy-guided KV-cache eviction system for 50% memory reduction in LLM inference, implemented in rtx-memory crate.

#### New Files
- **entropy_cache.rs**: Core entropy-guided eviction (~520 lines)
  - `EntropyMetrics` with attention/access entropy, token importance, cumulative attention
  - `EntropyTracker` for per-block entropy tracking with temporal decay
  - `EntropyConfig` with configurable thresholds and decay factors
  - `EntropyEvictionPolicy` enum: PureLowEntropy, AttentionWeighted, EntropyLru, Adaptive
  - Shannon entropy calculation from attention weight distributions
  - Eviction candidate selection with combined scoring (entropy + recency + frequency)

- **paged_attention.rs**: Page table management (~770 lines)
  - `PageTable` with logical-to-physical page mapping
  - `PagedAttentionConfig` with block size, num blocks, CoW support
  - `PhysicalPageInfo` tracking allocation state, entropy scores, access patterns
  - `SequencePages` for per-sequence page tracking with ref counting
  - `BlockTable` for batched attention kernel dispatch
  - Copy-on-Write support for beam search and prefix sharing
  - Memory pressure levels: Low, Medium, High, Critical

- **kv_cache.rs**: Full KV-cache allocator (~720 lines)
  - `KvCacheAllocator` integrating entropy tracking and paged attention
  - `KvCacheConfig` with page size, head dim, num heads, dtype, memory tiers
  - `KvCacheHandle` for safe sequence cache access
  - `KvDataType` enum: Float16, BFloat16, Float32, Int8 (quantized)
  - `KvMemoryTier` enum: Gpu, Cpu, Disk for tiered storage
  - `SequenceCache` with generation tracking and memory tier placement
  - Entropy-guided eviction with configurable policies
  - LRU fallback when entropy tracking disabled

#### Integration
- Updated `lib.rs` with module declarations and re-exports
- Compatible with existing `gpu_oom.rs` OOM recovery strategies
- 24 tests passing across all three modules

#### Key Algorithms (MorphKV-inspired)
- Shannon entropy: `-sum(p * log(p))` from attention weights
- Normalized entropy for [0,1] scoring
- Combined eviction score: `entropy_factor * 0.5 + recency * 0.3 + frequency * 0.2`
- Adaptive threshold scaling with memory pressure

---

### Added - Sparse Autoencoders (SAE) / Mechanistic Interpretability

Complete SAE module for LLM feature extraction and mechanistic interpretability, integrated into rtx-interpret crate.

#### New Files
- **sae/mod.rs**: Core SAE implementation
  - `SparseAutoencoder` struct with encoder/decoder weights
  - `SparsityType` enum: L1, TopK, JumpReLU, BatchTopK (Anthropic-style)
  - `SAEConfig` with expansion factor, sparsity settings, normalization
  - Forward pass with sparsity enforcement and loss computation
  - Dead neuron detection and activation tracking

- **sae/hooks.rs**: Layer hooking system
  - `ActivationHook` for capturing intermediate activations
  - `LayerHooks` for managing multiple hooks across layers
  - `BatchActivationCollector` for memory-efficient batch collection
  - Forward/backward hook support with gradient capture
  - Streaming mode for large model activation extraction

- **sae/training.rs**: SAE training utilities
  - `SAETrainer` with Adam optimizer and learning rate scheduling
  - Dead neuron detection and resampling (configurable interval)
  - Decoder normalization constraint enforcement
  - Warmup, cosine decay, and auxiliary loss computation
  - `TrainingHistory` for loss/sparsity tracking

- **sae/features.rs**: Feature analysis tools
  - `FeatureAnalyzer` for activation statistics
  - `FeatureStats`: activation frequency, mean/max values, sparsity
  - `TopActivation` tracking for max-activating examples
  - Co-activation matrix computation
  - `FeatureImportance` with multiple ranking methods
  - `SparsityStats` for L0/L1 norms, dead feature detection

#### Integration
- Updated `lib.rs` with sae module and re-exports
- Extends existing attribution/neuron analysis in rtx-interpret
- Compatible with rtx-tensor sparse tensor support (SparseCOO, SparseCSR)

---

### Added - SlideScope Pathology Demo

Complete GPU-accelerated stain separation demo for digital pathology, integrated into the RustyTorch++ Medical Demos Tauri application.

#### New Crates
- **slidescope-shared**: IPC types for Tauri communication
  - `SlideMetadata`, `NmfConfig`, `NmfResult`, `JobStatus`, `TileRequest/Response`
  - `SlidescopeStatus`, `SlideFilter`, `JobProgress`, `StainType`, `ImageFormat`
  - 19 tests passing

- **rtx-slidescope**: Core pathology processing crate
  - `nmf.rs` - CPU-based NMF with multiplicative updates
  - `optical_density.rs` - RGB to optical density conversion (Beer-Lambert law)
  - `stain_vectors.rs` - Macenko and Ruifrok stain estimation
  - `pyramid.rs` - Deep zoom tile pyramid generation
  - 22 tests passing

#### GPU Abstraction Layer
- `gpu/mod.rs` - `GpuBackend` trait with `CpuFallbackBackend`
- `gpu/cuda.rs` - CUDA backend using cudarc with custom NMF kernels
- `gpu/metal.rs` - Metal backend using wgpu with WGSL compute shaders
- `gpu_nmf.rs` - GPU-accelerated NMF processor
- 32 tests with Metal feature enabled

#### Server Integration
- `slidescope_service.rs` - Full service with slide import, tile serving, NMF processing
- Job queue management with progress tracking
- LRU tile cache for efficient deep zoom viewing

#### Tauri Commands (11 new)
- `slidescope_initialize`, `slidescope_import_slide`, `slidescope_list_slides`
- `slidescope_get_slide`, `slidescope_get_tile`, `slidescope_queue_processing`
- `slidescope_job_status`, `slidescope_get_result`, `slidescope_status`
- `slidescope_reset`, `slidescope_delete_slide`

#### Frontend
- `SlidescopeDemo.tsx` - React page with dual viewport for slide/stain viewing
- NMF configuration panel (components, iterations, GPU toggle)
- Progress tracking for processing jobs
- Added to demo gallery and App.tsx routing

---

## [December 11, 2025]

### Added - Apple Metal GPU Backend
- **Native Metal Support**: Complete Apple Metal GPU backend for Apple Silicon (M1/M2/M3/M4)
  - `metal_backend.rs` - Device discovery, buffer allocation, command encoding
  - `metal_compute.rs` - Shader compilation and pipeline management
  - `metal_blas/mod.rs` - MPS GEMM wrapper for matrix multiplication
  - `metal_ops.rs` - High-level tensor operation dispatch
- **Metal Shading Language (MSL) Kernels**:
  - `elementwise.metal` - Add, sub, mul, div, neg, abs, sqrt, exp, log, fma
  - `activations.metal` - ReLU, sigmoid, tanh, GELU, SiLU with forward/backward passes
  - `fourier.metal` - Sin/cos for Fourier features, positional encoding (PINN optimized)
  - `reductions.metal` - Sum, mean, max, min with threadgroup memory
- **Metal Performance Shaders (MPS)**: Hardware-accelerated GEMM (~7 TFLOPS on M1 Max)
- **Unified Memory**: Zero-copy CPU/GPU access via `MTLStorageModeShared`
- **Storage Integration**: `MetalGpu` variant in `StorageData` enum
- **Device Detection**: Real `metal_device_count()` using `MTLCreateSystemDefaultDevice`

### Changed
- **rtx-tensor/Cargo.toml**: Added objc2-metal, objc2-metal-performance-shaders dependencies
- **rtx-runtime/Cargo.toml**: Added objc2-metal dependencies for Metal backend
- **matrix_multiplication.rs**: Added Metal matmul dispatch using MPS
- **storage/core.rs**: Added Metal buffer accessors (`get_metal_data()`, `is_metal_native()`)
- **device.rs**: Implemented real Metal device detection for macOS
- **lib.rs**: Added metal_compute, metal_blas, metal_ops module exports

### Dependencies Added (macOS only)
- `objc2 = "0.6"` - Rust ObjC runtime bindings
- `objc2-metal = "0.3"` - Metal API bindings
- `objc2-metal-performance-shaders = "0.3"` - MPS bindings
- `objc2-foundation = "0.3"` - Foundation framework bindings

---

## [Previous] - October 26, 2025

### Removed (Honesty & Focus Improvements)
- **Quantum computing stub code** - Removed non-functional placeholder implementations
  - `rtx-transformers/src/revolutionary/quantum_types.rs` (162 lines of stubs)
  - Quantum Flash Attention variants (placeholders only)
  - Quantum pattern detection in fusion analyzer
- **Neuromorphic computing stub code** - Removed aspirational features
  - Neuromorphic Flash Attention variants
  - Neuromorphic test files and benchmarks
  - Spiking neural network stubs
- **Misleading documentation** - Removed unsubstantiated claims
  - "World's First Quantum-Classical-Neuromorphic" title
  - 30+ quantum/neuromorphic references in README
  - Performance claims without validation (1000x, ∞ advantages)
- **Test files for non-existent features**
  - `neuromorphic_integration_tests.rs`
  - `neuromorphic_efficiency_benchmark.rs`
  - `revolutionary_integration_tests.rs` (quantum-focused)
  - `hybrid_orchestrator_demo.rs` (quantum-focused)

### Changed
- **README.md** - Complete honesty overhaul
  - New title: "Production-Ready GPU-Accelerated ML Framework in Pure Rust"
  - Replaced aspirational claims with actual capabilities
  - Added honest status: "Core Infrastructure Stable"
  - Removed quantum/neuromorphic sections, examples, and claims
- **Module structure** - Cleaned revolutionary module
  - Renamed `HybridQuantumClassicalOrchestrator` → `HybridOrchestrator`
  - Removed quantum_types module exports
  - Focused on edge-classical orchestration only
- **Flash Attention variants** - Simplified to actual implementations
  - Removed quantum/neuromorphic placeholder modules
  - Clear documentation about supported variants

### Fixed (October 26, 2025 Session)
- **Device::Cpu bugs** - Fixed 16 critical bugs from incorrect find-and-replace
  - `device.rs`: Fixed is_cpu(), cpu() constructor, match arms
  - `serialization.rs`: Fixed device serialization
  - `matrix_multiplication.rs`: Fixed CPU device handling
- **Compilation errors** - Restored 11/11 core crates to compiling state
- **Rust 1.93 nightly** - Installed latest toolchain
- CUDA integration with cudarc 0.17.3
- Removed all PyTorch dependencies
- Implemented pure Rust tensor operations
- Fixed autograd compilation errors
- Fixed runtime stream issues
- Resolved tensor core operations
- Fixed cuBLAS batched operations
- Fixed cuDNN convolution descriptors
- Fixed cuSparse advanced operations
- Fixed cuSolver context issues
- Resolved character addition errors in tokenization
- Fixed visibility qualifiers in kernel modules
- Fixed cuda_launch_kernel references across modules
- Fixed Python binding issues in rtx-bindings
- Resolved PoolType imports in memory module

### Added
- **SESSION_PROGRESS_REPORT.md** - Detailed progress tracking
- **Honest assessment** in documentation
- Clear roadmap with realistic timeline
- Comprehensive error handling for CUDA operations
- Improved sparse tensor support
- Enhanced streaming capabilities
- Production monitoring improvements

### Migration from Rust 2024 Edition
- Migrated from RustaCUDA to cudarc
- Updated to Rust 2024 edition
- Consolidated compilation status reporting
- Reorganized documentation structure

## [Previous Versions]
See `docs/archive/legacy/` for historical development phases and changes.