rustyverse/rustytorch
2
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
402 Commits 1 Branch 0 Tags
ff7cc2d7396e38ae003a430f3ab8f2758e513dad
Commit Graph
3 Commits
Author SHA1 Message Date
osobhandClaude Sonnet 5 4aaa36a57a style: cargo fmt --workspace (whitespace/wrapping only, no semantic change) 
Whole-workspace rustfmt pass picked up while iterating on Mamba GPU
backward work. Verified formatting-only via diff sampling; no logic
changed.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
 2026-08-10 07:09:36 -07:00
Omar SobhandClaude Sonnet 4.6 d6769ef641 feat(perf): GPU perf batch 4 — SmoothQuant INT8 forward, varlen FA, inference graph capture
GPU Tests / Check GPU Availability (push) Successful in 1s
Details
GPU Tests / CUDA Tests (12.1) (push) Has been skipped
Details
GPU Tests / CUDA Tests (11.8) (push) Has been skipped
Details
Documentation / Build API Documentation (push) Failing after 8s
Details
CI / Build (ubuntu-latest) (push) Failing after 8s
Details
CI / Build CPU-Only (Explicit) (push) Failing after 9s
Details
Documentation / Build User Guide (push) Successful in 10s
Details
Performance Benchmarks / Run Benchmarks (push) Successful in 1m10s
Details
CI / Format Check (push) Failing after 10s
Details
CI / Clippy Check (push) Failing after 16s
Details
CI / Build (macos-latest) (push) Failing after 49s
Details
CI / Test (macos-latest) (push) Has been skipped
Details
CI / Test (ubuntu-latest) (push) Has been skipped
Details
CI / Python Bindings (maturin) (macos-latest) (push) Has been skipped
Details
CI / Python Bindings (maturin) (ubuntu-latest) (push) Has been skipped
Details
CI / WASM Build + Size Check (push) Has been skipped
Details
CI / Distributed Training Tests (push) Has been skipped
Details
CI / CI Success (push) Failing after 0s
Details
GPU Tests / Metal Tests (push) Has been skipped
Details 
SmoothQuant INT8 linear forward (rtx-compress)
- `advanced.rs`: `SmoothQuantizedLayer::forward_raw()` — smooth activations ÷ scales,
  INT8-quantize both sides (range −127…127 symmetric), INT8 GEMM w/ i32 accumulation,
  dequantize: acc * act_scale * weight_scale; `out_features()` / `in_features()` helpers
- `int8_matmul.rs`: `int8_matvec` + `int8_gemm` (i32 accumulation); 7 unit tests
- 4 forward_raw tests: shape, identity layer, scale effect, manual verification
- 11 new tests; total 112 pass

Variable-length packed flash attention (rtx-flash-attention) [commit 80d7c7f]
- `flash_attention_varlen.cu`: WGMMA-compatible CUDA kernel, BLOCK_Q/K=64, block=(128,1,1),
  grid=(ceil(max_seqlen/64), heads, 1); linear cu_seqlens scan for sequence-to-block
  mapping; 24KB shared memory (3 × 64 × 64 × 2 bytes); early-exit for past-end blocks
- `flash_varlen_forward.rs`: `varlen_attention_cpu` O(n²) reference + `#[cfg(cuda)]`
  `FlashVarlenKernel` NVRTC wrapper; `SdpaBackend::VarLen` added to backend_selector
- 8 CPU tests: single-seq matches regular attn, two seqs independent, causal mask,
  softmax sums to 1, empty sequence handled, output shape; total 50 pass

Inference CUDA graph capture (rtx-inference)
- `inference_graph.rs`: `InferenceGraphCapture` + `StepMode` {Warmup, Capture, Replay};
  state machine: N warmup steps → capture once → replay forever; `check_static_shape()`
  invalidates on batch/step change; `record_capture(graph_id)` stores graph
- `batch_processor.rs`: `graph_capture: Mutex<InferenceGraphCapture>` field (#[cfg(cuda)]);
  `advance()` wired at line 852 in `execute_batch_inference`; stream TODO matches
  training_loop.rs pattern; config gains `enable_decode_graphs`/`graph_warmup_steps`
- 8 pure-logic tests; total 93 pass; 4 integration test literals fixed

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
 2026-06-27 01:11:08 +00:00
redclawsystems 4d88dc0584 Initial commit 2026-03-04 00:08:42 +00:00