Omar SobhandClaude Sonnet 4.6
1eb89c5b2b
CI / Build CPU-Only (Explicit) (push) Failing after 8s
CI / Clippy Check (push) Failing after 12s
GPU Tests / Check GPU Availability (push) Successful in 0s
CI / Format Check (push) Failing after 14s
Performance Benchmarks / Run Benchmarks (push) Failing after 15s
Documentation / Build User Guide (push) Successful in 6s
Documentation / Build API Documentation (push) Failing after 16s
GPU Tests / CUDA Tests (11.8) (push) Has been skipped
GPU Tests / CUDA Tests (12.1) (push) Has been skipped
CI / Build (ubuntu-latest) (push) Failing after 1m20s
CI / Build (macos-latest) (push) Failing after 1m26s
CI / Test (macos-latest) (push) Has been skipped
CI / Test (ubuntu-latest) (push) Has been skipped
CI / Python Bindings (maturin) (macos-latest) (push) Has been skipped
CI / Python Bindings (maturin) (ubuntu-latest) (push) Has been skipped
CI / WASM Build + Size Check (push) Has been skipped
CI / Distributed Training Tests (push) Has been skipped
CI / CI Success (push) Failing after 0s
GPU Tests / Metal Tests (push) Has been skipped
Item 1 — CUDA Graphs wiring (rtx-transformers)
- Added `enable_cuda_graphs: bool` (default false) + `cuda_graph_warmup_iters: usize`
(default 3) to `TrainingConfig`
- Wired 3-phase state machine into `training_loop.rs` (warmup → capture → replay)
gated on `#[cfg(feature = "cuda")]`; stream plumbing stubbed with TODO pending
`CudaStreamHandle` threading
Item 2 — FP8 E4M3/E5M2 training infrastructure (rtx-tensor, rtx-transformers)
- `fp8_cast.cu`: dual-path CUDA kernels — SM_89+ uses `<cuda_fp8.h>` native
`__nv_cvt_*` intrinsics; older SM uses software bit-manipulation fallback
- `fp8_cast.rs`: host-side CPU casting + `#[cfg(feature = "cuda")]` GPU stubs
- `fp8_gemm.rs`: bit-accurate E4M3 decoder/encoder, BF16 round-trip utils, CPU
reference matmul with cuBLASLt GPU path documented inline; 12 unit tests
- `training_config.rs`: `fp8_training: bool`, `fp8_e4m3_forward: bool`
- `linear.rs` (modular): `fp8_mode: bool` field + forward dispatch stub
- build.rs: registers `fp8_cast.cu` alongside existing `element_wise.cu`
- 22 FP8 unit tests — all pass
Item 3 — FlashAttention-3 Blackwell (WGMMA + TMA + warp specialization)
- `flash_attention_v3_forward.cu`: SM_90+ warp-specialised producer/consumer
kernel (producer TMA-loads K/V tiles, consumers run WMMA as portable WGMMA
proxy); SM_89+ FP8 header path; SM<90 standard FA2-style WMMA fallback
- `flash_v3_forward.rs`: NVRTC wrapper (`compile_ptx` via `include_str!`),
`FlashV3ForwardKernel::new/is_supported/forward`; 6 unit tests
- `backend_selector.rs`: `SdpaBackend::FlashAttentionV3`, `for_compute_capability`,
`supports_flash_v3` (major >= 9), FA3 scoring (0.98/0.90/0.70), 2× speedup estimate
- `kernels/simple.rs`: `v3_kernel: Option<FlashV3ForwardKernel>` in `FlashCudaKernels`
- Fixed pre-existing `Device::Cpu` cfg-gate bug in `tensor/creation.rs`
- 8 new FA3 backend tests + 6 kernel unit tests; 50 total pass
Item 4 — SnapKV attention-score eviction + prefix caching (rtx-inference, rtx-serving-api)
- `prefix_index.rs`: `PrefixIndex` with 8MB Zobrist hash table (Knuth MMIX LCG seed),
`compute_hash/lookup/insert/remove/remove_page`; 10 unit tests
- `eviction.rs`: `AttentionScoreEviction` struct — `accumulate_scores` +
`select_evict_positions` (retain top keep_ratio + last recent_window); 7 unit tests
- `types.rs`: `EvictionPolicy::AttentionScore { keep_ratio, recent_window }` +
`KvCacheConfig::enable_prefix_caching`
- `paged_kv_cache.rs`: `prefix_index: Option<PrefixIndex>` + `lookup_prefix /
register_prefix / unregister_prefix_page / prefix_caching_enabled` methods
- `config.rs` (serving-api): `enable_prefix_sharing: true` (was false),
`snapkv_keep_ratio: 0.6`, `snapkv_recent_window: 32`
- Fixed 12 pre-existing test errors (spurious `.await` on sync constructors)
- 17 SnapKV/prefix tests pass
Total: 918 lib tests pass across rtx-tensor, rtx-flash-attention, rtx-transformers,
rtx-inference. Zero new failures.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>