1 Commits

Include renames

Author	SHA1	Message	Date
Omar SobhandClaude Sonnet 4.6	d6769ef641	feat(perf): GPU perf batch 4 — SmoothQuant INT8 forward, varlen FA, inference graph capture ... SmoothQuant INT8 linear forward (rtx-compress) - `advanced.rs`: `SmoothQuantizedLayer::forward_raw()` — smooth activations ÷ scales, INT8-quantize both sides (range −127…127 symmetric), INT8 GEMM w/ i32 accumulation, dequantize: acc * act_scale * weight_scale; `out_features()` / `in_features()` helpers - `int8_matmul.rs`: `int8_matvec` + `int8_gemm` (i32 accumulation); 7 unit tests - 4 forward_raw tests: shape, identity layer, scale effect, manual verification - 11 new tests; total 112 pass Variable-length packed flash attention (rtx-flash-attention) [commit 80d7c7f] - `flash_attention_varlen.cu`: WGMMA-compatible CUDA kernel, BLOCK_Q/K=64, block=(128,1,1), grid=(ceil(max_seqlen/64), heads, 1); linear cu_seqlens scan for sequence-to-block mapping; 24KB shared memory (3 × 64 × 64 × 2 bytes); early-exit for past-end blocks - `flash_varlen_forward.rs`: `varlen_attention_cpu` O(n²) reference + `#[cfg(cuda)]` `FlashVarlenKernel` NVRTC wrapper; `SdpaBackend::VarLen` added to backend_selector - 8 CPU tests: single-seq matches regular attn, two seqs independent, causal mask, softmax sums to 1, empty sequence handled, output shape; total 50 pass Inference CUDA graph capture (rtx-inference) - `inference_graph.rs`: `InferenceGraphCapture` + `StepMode` {Warmup, Capture, Replay}; state machine: N warmup steps → capture once → replay forever; `check_static_shape()` invalidates on batch/step change; `record_capture(graph_id)` stores graph - `batch_processor.rs`: `graph_capture: Mutex<InferenceGraphCapture>` field (#[cfg(cuda)]); `advance()` wired at line 852 in `execute_batch_inference`; stream TODO matches training_loop.rs pattern; config gains `enable_decode_graphs`/`graph_warmup_steps` - 8 pure-logic tests; total 93 pass; 4 integration test literals fixed Co-Authored-By: Claude Sonnet 4.6 <[email protected]>	2026-06-27 01:11:08 +00:00