rustyverse/rustytorch
2
0
Fork 0
Code Issues Pull Requests Actions 28 Packages Projects Releases Wiki Activity
427 Commits 1 Branch 0 Tags
7f144e00056fe4cc3fcf26b420c713add475353e
Commit Graph
2 Commits
Author SHA1 Message Date
Omar SobhandClaude Sonnet 4.6 082a50e3a0 feat(perf): GPU perf batch 1 — wire GPU execution paths for FP8, FA3, CUDA Graphs, SnapKV
CI / Build (ubuntu-latest) (push) Failing after 8s
Details
Performance Benchmarks / Run Benchmarks (push) Successful in 8s
Details
CI / Clippy Check (push) Failing after 8s
Details
CI / Build CPU-Only (Explicit) (push) Failing after 8s
Details
Documentation / Build API Documentation (push) Failing after 7s
Details
CI / Format Check (push) Failing after 9s
Details
GPU Tests / Check GPU Availability (push) Successful in 0s
Details
Documentation / Build User Guide (push) Successful in 7s
Details
GPU Tests / CUDA Tests (11.8) (push) Has been skipped
Details
GPU Tests / CUDA Tests (12.1) (push) Has been skipped
Details
CI / Build (macos-latest) (push) Failing after 14s
Details
CI / Test (macos-latest) (push) Has been skipped
Details
CI / Test (ubuntu-latest) (push) Has been skipped
Details
CI / Python Bindings (maturin) (macos-latest) (push) Has been skipped
Details
CI / Python Bindings (maturin) (ubuntu-latest) (push) Has been skipped
Details
CI / WASM Build + Size Check (push) Has been skipped
Details
CI / Distributed Training Tests (push) Has been skipped
Details
CI / CI Success (push) Failing after 0s
Details
GPU Tests / Metal Tests (push) Has been skipped
Details 
FP8 GPU FFI (rtx-tensor)
- `fp8_cast.rs`: replaced `not_implemented` stubs with real cudarc 0.18.2 PTX launches;
  `cast_bf16_to_fp8_e4m3` and `cast_fp8_e4m3_to_bf16` now dispatch to NVCC-compiled
  `fp8_cast.ptx` via `LazyLock` module cache, matching the `inplace_ops` pattern
- `build.rs`: `create_dummy_ptx` now emits `fp8_cast.ptx` alongside `element_wise.ptx`
  so `include_str!` resolves cleanly when NVCC is absent

FlashAttention-3 typed kernel launch (rtx-flash-attention)
- `flash_v3_forward.rs`: `forward()` now takes typed `CudaSlice<bf16>` Q/K/V/O + `CudaSlice<f32>`
  LSE buffer; dispatches via `stream.launch_builder` with block_dim=(128,1,1),
  grid_dim=(ceil(seq_len/64), batch*heads, 1), shared_mem_bytes=0 (PTX metadata-resolved)
- `simple.rs`: added `has_flash_v3()` + `flash_attention_v3_forward_raw()` dispatch
- `Cargo.toml`: `half` added as optional cuda-gated dependency

CUDA Graphs stream threading (rtx-transformers)
- `training_loop.rs`: added `cuda_stream: Option<CudaStreamHandle>` field; `set_cuda_backend()`
  now creates a non-default capture stream; capture step calls real `begin_capture(stream)` +
  `end_capture(stream)`; added `set_cuda_stream()` override; replay unchanged (no stream needed)

SnapKV + prefix cache BatchScheduler wiring (rtx-inference)
- `scheduler.rs`: added `prefix_hit_pages: Option<Vec<PageId>>` + `evicted_positions: Vec<usize>`
  to `SchedulerRequest`; `BatchScheduler` gains `kv_cache` + `snapkv_eviction` fields;
  `submit_request` does non-blocking `try_lock` prefix lookup; added `notify_prefill_complete`
  (registers prefix + runs `select_evict_positions`), `set_kv_cache`, `set_snapkv_eviction`,
  `get_evicted_positions`, `get_prefix_hit_pages` — +5 new integration tests

Test results: 22 + 42 + 75 + 102 = 241 tests, 0 failures

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
 2026-06-26 20:39:58 +00:00
redclawsystems 4d88dc0584 Initial commit 2026-03-04 00:08:42 +00:00