# RustyTorch++ CUDA Work Status

**Date**: January 7, 2026
**Last Updated Before Reboot**

---

## Completed Work

### Phase 1: CUDA Environment Setup (DONE)
- Fixed CUDA 13.0 installation
- Reinstalled libcublas-13-0 for cuBLASLt linking
- Verified rtx-backend-cuda: **42 tests passed**
- Verified rtx-tensor: **622 tests passed**

### Phase 2: MX GPU Kernels (DONE)
- Created CUDA kernel file: `crates/training/rtx-compress/src/quantization/cuda_kernels/mx_kernels.cu`
- Created build.rs for rtx-compress CUDA compilation
- Wired up GPU kernels in `mx_gpu_kernels.rs`
- Tests passing

### Phase 3: cuSPARSELt Integration (DONE)
- Fixed struct sizes in `types.rs` (512 bytes with 16-byte alignment, not 11024)
- Fixed status codes to match cuSPARSE header (NotSupported = 10)
- Added real FFI bindings with `#[link(name = "cusparseLt")]` in `ffi.rs`
- Updated `spmm.rs` to use cudarc 0.18 stream-based API
- Added `cusparselt_link` feature to Cargo.toml
- Added `link_cusparselt()` function to build.rs
- **10 cuSPARSELt tests pass**
- Library detected as available: `cuSPARSELt available: true`

### Phase 4: Benchmarks (PARTIAL)
- Ran CPU-only benchmarks (GPU benchmarks blocked by driver issue)
- CPU matmul results:
  - 128x128: 1.05 ms, 4.0 GFLOPS
  - 256x256: 8.82 ms, 3.8 GFLOPS
  - 512x512: 105 ms, 2.6 GFLOPS
  - 1024x1024: 2496 ms, 0.9 GFLOPS
  - 2048x2048: 29575 ms, 0.6 GFLOPS
- 2:4 Sparsity SpMM (CPU fallback): 1.8x compression achieved

---

## Current Blocker

**Driver/cudarc Incompatibility**

cudarc 0.18 requires the `cuDevSmResourceSplit` CUDA driver API which is not available in driver 580.105.08:

```
thread panicked at cudarc-0.18.2/src/driver/sys/mod.rs:21655:18:
Expected symbol in library: DlSym {
  desc: "/lib/x86_64-linux-gnu/libcuda.so: undefined symbol: cuDevSmResourceSplit"
}
```

**Solution**: Upgrade to NVIDIA driver 590.x

---

## Pending: Driver Upgrade

### Before Reboot - Run These Commands:
```bash
# Add NVIDIA 590 PPA
sudo add-apt-repository -y ppa:jacobmartin/nv-graphics-2
sudo apt update

# Install driver 590
sudo apt install nvidia-driver-590

# Reboot
sudo reboot
```

### After Reboot - Tell Claude:

> Continue from CUDA_WORK_STATUS.md - verify driver 590 installation and run GPU benchmarks

---

## After Reboot Checklist

### 1. Verify Driver Installation
```bash
nvidia-smi
# Expected: Driver Version 590.x

nvcc --version
# Expected: CUDA 13.1.x
```

### 2. Test cudarc Compatibility
```bash
cd /home/osobh/data/HPC-AI/rustytorch
cargo test -p rtx-tensor --features cuda --lib "cusparelt::handle" -- --nocapture
# Should NOT panic on cuDevSmResourceSplit
```

### 3. Run Full GPU Tests
```bash
cargo test -p rtx-tensor --features cuda -- --test-threads=1
cargo test -p rtx-backend-cuda --features cuda -- --test-threads=1
```

### 4. Run GPU Benchmarks
```bash
cargo run -p rtx-tensor --release --example quick_benchmark --features cuda
```

### 5. Test cuSPARSELt GPU Path
```bash
cargo test -p rtx-tensor --features cuda,cusparselt_link --lib "cusparelt" -- --nocapture
```

---

## Key Files Modified

| File | Changes |
|------|---------|
| `crates/core/rtx-tensor/src/cusparelt/types.rs` | Fixed struct sizes to 512 bytes |
| `crates/core/rtx-tensor/src/cusparelt/ffi.rs` | Added real FFI bindings |
| `crates/core/rtx-tensor/src/cusparelt/spmm.rs` | Updated to cudarc 0.18 API |
| `crates/core/rtx-tensor/src/cusparelt/handle.rs` | Added panic catch for driver compat |
| `crates/core/rtx-tensor/build.rs` | Added link_cusparselt() |
| `crates/core/rtx-tensor/Cargo.toml` | Added cusparselt_link feature |
| `crates/training/rtx-compress/src/quantization/cuda_kernels/mx_kernels.cu` | NEW - MX CUDA kernels |
| `crates/training/rtx-compress/build.rs` | NEW - CUDA kernel compilation |
| `crates/core/rtx-tensor/examples/quick_benchmark.rs` | NEW - Quick perf benchmark |

---

## Rollback Plan (If Driver 590 Causes Issues)

```bash
# Remove driver 590
sudo apt remove nvidia-driver-590

# Reinstall driver 580
sudo apt install nvidia-driver-580

# Remove PPA
sudo add-apt-repository --remove ppa:jacobmartin/nv-graphics-2

# Reboot
sudo reboot
```

---

## Hardware Info

- **GPU**: NVIDIA GeForce RTX 3050 Ti Laptop GPU (4GB VRAM)
- **Architecture**: Ampere (SM 86)
- **OS**: Ubuntu 24.04.3 LTS
- **Current Driver**: 580.105.08
- **Target Driver**: 590.x
- **CUDA Toolkit**: 13.1.80
- **cuSPARSELt**: Installed at `/usr/lib/x86_64-linux-gnu/libcusparseLt/13/`