rustytorch/CUDA_WORK_STATUS.md

RustyTorch++ CUDA Work Status

Date: January 7, 2026 Last Updated Before Reboot

---

Completed Work

Phase 1: CUDA Environment Setup (DONE)

• Fixed CUDA 13.0 installation
• Reinstalled libcublas-13-0 for cuBLASLt linking
• Verified rtx-backend-cuda: 42 tests passed
• Verified rtx-tensor: 622 tests passed

Phase 2: MX GPU Kernels (DONE)

• Created CUDA kernel file: crates/training/rtx-compress/src/quantization/cuda_kernels/mx_kernels.cu
• Created build.rs for rtx-compress CUDA compilation
• Wired up GPU kernels in mx_gpu_kernels.rs
• Tests passing

Phase 3: cuSPARSELt Integration (DONE)

• Fixed struct sizes in types.rs (512 bytes with 16-byte alignment, not 11024)
• Fixed status codes to match cuSPARSE header (NotSupported = 10)
• Added real FFI bindings with #[link(name = "cusparseLt")] in ffi.rs
• Updated spmm.rs to use cudarc 0.18 stream-based API
• Added cusparselt_link feature to Cargo.toml
• Added link_cusparselt() function to build.rs
• 10 cuSPARSELt tests pass
• Library detected as available: cuSPARSELt available: true

Phase 4: Benchmarks (PARTIAL)

• Ran CPU-only benchmarks (GPU benchmarks blocked by driver issue)
• CPU matmul results:
  • 128x128: 1.05 ms, 4.0 GFLOPS
  • 256x256: 8.82 ms, 3.8 GFLOPS
  • 512x512: 105 ms, 2.6 GFLOPS
  • 1024x1024: 2496 ms, 0.9 GFLOPS
  • 2048x2048: 29575 ms, 0.6 GFLOPS
• 2:4 Sparsity SpMM (CPU fallback): 1.8x compression achieved

---

Current Blocker

Driver/cudarc Incompatibility

cudarc 0.18 requires the cuDevSmResourceSplit CUDA driver API which is not available in driver 580.105.08:

thread panicked at cudarc-0.18.2/src/driver/sys/mod.rs:21655:18:
Expected symbol in library: DlSym {
  desc: "/lib/x86_64-linux-gnu/libcuda.so: undefined symbol: cuDevSmResourceSplit"
}

Solution: Upgrade to NVIDIA driver 590.x

---

Pending: Driver Upgrade

Before Reboot - Run These Commands:

# Add NVIDIA 590 PPA
sudo add-apt-repository -y ppa:jacobmartin/nv-graphics-2
sudo apt update

# Install driver 590
sudo apt install nvidia-driver-590

# Reboot
sudo reboot

After Reboot - Tell Claude:

Continue from CUDA_WORK_STATUS.md - verify driver 590 installation and run GPU benchmarks

---

After Reboot Checklist

1. Verify Driver Installation

nvidia-smi
# Expected: Driver Version 590.x

nvcc --version
# Expected: CUDA 13.1.x

2. Test cudarc Compatibility

cd /home/osobh/data/HPC-AI/rustytorch
cargo test -p rtx-tensor --features cuda --lib "cusparelt::handle" -- --nocapture
# Should NOT panic on cuDevSmResourceSplit

3. Run Full GPU Tests

cargo test -p rtx-tensor --features cuda -- --test-threads=1
cargo test -p rtx-backend-cuda --features cuda -- --test-threads=1

4. Run GPU Benchmarks

cargo run -p rtx-tensor --release --example quick_benchmark --features cuda

5. Test cuSPARSELt GPU Path

cargo test -p rtx-tensor --features cuda,cusparselt_link --lib "cusparelt" -- --nocapture

---

Key Files Modified

File	Changes
crates/core/rtx-tensor/src/cusparelt/types.rs	Fixed struct sizes to 512 bytes
crates/core/rtx-tensor/src/cusparelt/ffi.rs	Added real FFI bindings
crates/core/rtx-tensor/src/cusparelt/spmm.rs	Updated to cudarc 0.18 API
crates/core/rtx-tensor/src/cusparelt/handle.rs	Added panic catch for driver compat
crates/core/rtx-tensor/build.rs	Added link_cusparselt()
crates/core/rtx-tensor/Cargo.toml	Added cusparselt_link feature
crates/training/rtx-compress/src/quantization/cuda_kernels/mx_kernels.cu	NEW - MX CUDA kernels
crates/training/rtx-compress/build.rs	NEW - CUDA kernel compilation
crates/core/rtx-tensor/examples/quick_benchmark.rs	NEW - Quick perf benchmark

---

Rollback Plan (If Driver 590 Causes Issues)

# Remove driver 590
sudo apt remove nvidia-driver-590

# Reinstall driver 580
sudo apt install nvidia-driver-580

# Remove PPA
sudo add-apt-repository --remove ppa:jacobmartin/nv-graphics-2

# Reboot
sudo reboot

---

Hardware Info

• GPU: NVIDIA GeForce RTX 3050 Ti Laptop GPU (4GB VRAM)
• Architecture: Ampere (SM 86)
• OS: Ubuntu 24.04.3 LTS
• Current Driver: 580.105.08
• Target Driver: 590.x
• CUDA Toolkit: 13.1.80
• cuSPARSELt: Installed at /usr/lib/x86_64-linux-gnu/libcusparseLt/13/