161 lines
4.3 KiB
Markdown
161 lines
4.3 KiB
Markdown
# RustyTorch++ CUDA Work Status
|
|
|
|
**Date**: January 7, 2026
|
|
**Last Updated Before Reboot**
|
|
|
|
---
|
|
|
|
## Completed Work
|
|
|
|
### Phase 1: CUDA Environment Setup (DONE)
|
|
- Fixed CUDA 13.0 installation
|
|
- Reinstalled libcublas-13-0 for cuBLASLt linking
|
|
- Verified rtx-backend-cuda: **42 tests passed**
|
|
- Verified rtx-tensor: **622 tests passed**
|
|
|
|
### Phase 2: MX GPU Kernels (DONE)
|
|
- Created CUDA kernel file: `crates/training/rtx-compress/src/quantization/cuda_kernels/mx_kernels.cu`
|
|
- Created build.rs for rtx-compress CUDA compilation
|
|
- Wired up GPU kernels in `mx_gpu_kernels.rs`
|
|
- Tests passing
|
|
|
|
### Phase 3: cuSPARSELt Integration (DONE)
|
|
- Fixed struct sizes in `types.rs` (512 bytes with 16-byte alignment, not 11024)
|
|
- Fixed status codes to match cuSPARSE header (NotSupported = 10)
|
|
- Added real FFI bindings with `#[link(name = "cusparseLt")]` in `ffi.rs`
|
|
- Updated `spmm.rs` to use cudarc 0.18 stream-based API
|
|
- Added `cusparselt_link` feature to Cargo.toml
|
|
- Added `link_cusparselt()` function to build.rs
|
|
- **10 cuSPARSELt tests pass**
|
|
- Library detected as available: `cuSPARSELt available: true`
|
|
|
|
### Phase 4: Benchmarks (PARTIAL)
|
|
- Ran CPU-only benchmarks (GPU benchmarks blocked by driver issue)
|
|
- CPU matmul results:
|
|
- 128x128: 1.05 ms, 4.0 GFLOPS
|
|
- 256x256: 8.82 ms, 3.8 GFLOPS
|
|
- 512x512: 105 ms, 2.6 GFLOPS
|
|
- 1024x1024: 2496 ms, 0.9 GFLOPS
|
|
- 2048x2048: 29575 ms, 0.6 GFLOPS
|
|
- 2:4 Sparsity SpMM (CPU fallback): 1.8x compression achieved
|
|
|
|
---
|
|
|
|
## Current Blocker
|
|
|
|
**Driver/cudarc Incompatibility**
|
|
|
|
cudarc 0.18 requires the `cuDevSmResourceSplit` CUDA driver API which is not available in driver 580.105.08:
|
|
|
|
```
|
|
thread panicked at cudarc-0.18.2/src/driver/sys/mod.rs:21655:18:
|
|
Expected symbol in library: DlSym {
|
|
desc: "/lib/x86_64-linux-gnu/libcuda.so: undefined symbol: cuDevSmResourceSplit"
|
|
}
|
|
```
|
|
|
|
**Solution**: Upgrade to NVIDIA driver 590.x
|
|
|
|
---
|
|
|
|
## Pending: Driver Upgrade
|
|
|
|
### Before Reboot - Run These Commands:
|
|
```bash
|
|
# Add NVIDIA 590 PPA
|
|
sudo add-apt-repository -y ppa:jacobmartin/nv-graphics-2
|
|
sudo apt update
|
|
|
|
# Install driver 590
|
|
sudo apt install nvidia-driver-590
|
|
|
|
# Reboot
|
|
sudo reboot
|
|
```
|
|
|
|
### After Reboot - Tell Claude:
|
|
|
|
> Continue from CUDA_WORK_STATUS.md - verify driver 590 installation and run GPU benchmarks
|
|
|
|
---
|
|
|
|
## After Reboot Checklist
|
|
|
|
### 1. Verify Driver Installation
|
|
```bash
|
|
nvidia-smi
|
|
# Expected: Driver Version 590.x
|
|
|
|
nvcc --version
|
|
# Expected: CUDA 13.1.x
|
|
```
|
|
|
|
### 2. Test cudarc Compatibility
|
|
```bash
|
|
cd /home/osobh/data/HPC-AI/rustytorch
|
|
cargo test -p rtx-tensor --features cuda --lib "cusparelt::handle" -- --nocapture
|
|
# Should NOT panic on cuDevSmResourceSplit
|
|
```
|
|
|
|
### 3. Run Full GPU Tests
|
|
```bash
|
|
cargo test -p rtx-tensor --features cuda -- --test-threads=1
|
|
cargo test -p rtx-backend-cuda --features cuda -- --test-threads=1
|
|
```
|
|
|
|
### 4. Run GPU Benchmarks
|
|
```bash
|
|
cargo run -p rtx-tensor --release --example quick_benchmark --features cuda
|
|
```
|
|
|
|
### 5. Test cuSPARSELt GPU Path
|
|
```bash
|
|
cargo test -p rtx-tensor --features cuda,cusparselt_link --lib "cusparelt" -- --nocapture
|
|
```
|
|
|
|
---
|
|
|
|
## Key Files Modified
|
|
|
|
| File | Changes |
|
|
|------|---------|
|
|
| `crates/core/rtx-tensor/src/cusparelt/types.rs` | Fixed struct sizes to 512 bytes |
|
|
| `crates/core/rtx-tensor/src/cusparelt/ffi.rs` | Added real FFI bindings |
|
|
| `crates/core/rtx-tensor/src/cusparelt/spmm.rs` | Updated to cudarc 0.18 API |
|
|
| `crates/core/rtx-tensor/src/cusparelt/handle.rs` | Added panic catch for driver compat |
|
|
| `crates/core/rtx-tensor/build.rs` | Added link_cusparselt() |
|
|
| `crates/core/rtx-tensor/Cargo.toml` | Added cusparselt_link feature |
|
|
| `crates/training/rtx-compress/src/quantization/cuda_kernels/mx_kernels.cu` | NEW - MX CUDA kernels |
|
|
| `crates/training/rtx-compress/build.rs` | NEW - CUDA kernel compilation |
|
|
| `crates/core/rtx-tensor/examples/quick_benchmark.rs` | NEW - Quick perf benchmark |
|
|
|
|
---
|
|
|
|
## Rollback Plan (If Driver 590 Causes Issues)
|
|
|
|
```bash
|
|
# Remove driver 590
|
|
sudo apt remove nvidia-driver-590
|
|
|
|
# Reinstall driver 580
|
|
sudo apt install nvidia-driver-580
|
|
|
|
# Remove PPA
|
|
sudo add-apt-repository --remove ppa:jacobmartin/nv-graphics-2
|
|
|
|
# Reboot
|
|
sudo reboot
|
|
```
|
|
|
|
---
|
|
|
|
## Hardware Info
|
|
|
|
- **GPU**: NVIDIA GeForce RTX 3050 Ti Laptop GPU (4GB VRAM)
|
|
- **Architecture**: Ampere (SM 86)
|
|
- **OS**: Ubuntu 24.04.3 LTS
|
|
- **Current Driver**: 580.105.08
|
|
- **Target Driver**: 590.x
|
|
- **CUDA Toolkit**: 13.1.80
|
|
- **cuSPARSELt**: Installed at `/usr/lib/x86_64-linux-gnu/libcusparseLt/13/`
|