6 Commits

Include renames

Author	SHA1	Message	Date
osobhandClaude Sonnet 5	4aaa36a57a	style: cargo fmt --workspace (whitespace/wrapping only, no semantic change) ... Whole-workspace rustfmt pass picked up while iterating on Mamba GPU backward work. Verified formatting-only via diff sampling; no logic changed. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-08-10 07:09:36 -07:00
osobhandClaude Fable 5	64ade03ab9	fix(production): wire real inference path through engine, serving, and streaming ... - rtx-inference: sample_next_token now copies the actual logits from the forward pass (Tensor::to_vec, last-token slice) instead of sampling from a fabricated all-zero vector; request metrics report measured queue/processing times instead of hardcoded constants. - rtx-serving-api: depends on rtx-inference; /v1/completions dispatches to a shared InferenceEngine (byte-level tokenization until a real tokenizer is threaded through) and returns 503 when no engine is loaded instead of mock text. ServingServer::with_engine attaches one. - rtx-streaming: depends on rtx-inference; generate_tokens delegates to an attached backend engine and errors without one instead of emitting "token_N" placeholders; tokenization is byte-level, not position-mod. - speculative decoding: document the orchestration (speculative/) vs implementation (medusa.rs/lookahead.rs) layering; CLAUDE.md no longer claims a standalone rtx-speculative-decoding crate. Co-Authored-By: Claude Fable 5 <[email protected]>	2026-07-09 19:05:27 -07:00
Omar SobhandClaude Sonnet 4.6	ef3cdb1e1a	feat(batch9): token merging (ToMe), grad accum per-step norm, speculative streaming ... - Token Merging (ToMe, arXiv:2210.09461): bipartite soft matching via priority-queue second-chance loop; apply_merge/apply_unmerge; TokenMergingLayer::forward(); 17 unit tests + 2 doctests; 75% merge at r=32/seq=64 - Gradient accumulation per-step normalization: NormalizationStrategy {EndOfAccumulation, PerStep, None}; with_per_step_normalization(); compute_gradient_norm() L2 norm; PerStep divides by fixed accumulation_steps before add (not end-of-batch); 11 tests including equivalence proof vs EndOfAccumulation - Speculative streaming: SpeculativeStreamer + mpsc::Receiver<StreamedToken>; notify_step() sends accepted draft tokens + optional continuation immediately; StreamStats with Welford online mean latency; collect_stream() test helper; 17 async tests Co-Authored-By: Claude Sonnet 4.6 <[email protected]>	2026-06-27 04:38:20 +00:00
Omar SobhandClaude Sonnet 4.6	0e1d6a74b6	feat(batch6): windowed acceptance metrics, KV INT8 quant, col/row-parallel linear ... Windowed acceptance rate (rtx-inference/speculative): - WindowedAcceptanceTracker: O(1) VecDeque sliding window, p50/p95/min/max - AcceptanceTrend enum (Rising/Falling/Stable, ±0.05 threshold) - AcceptanceDashboard aggregator; wired into PerformanceMetrics::update() and dashboard(); 13 tests KV cache INT8 quantization (rtx-inference/cache): - KvCacheQuantMode { None, Int8 { scale_per_token }, Fp8E4M3 } enum - KvQuantizer::encode/decode: symmetric per-block INT8 (scale=max_abs/127) gives 4× compression vs f32; Fp8E4M3 CPU proxy, GPU path reserved - QuantizedKvBlock carries data+scale+mode; KvCacheConfig::quant_mode defaulting to None; 14 tests ColParallel + RowParallel linear (rtx-distributed): - ColParallelLinear: shards weight rows across TP ranks, forward_cpu() batch matmul + per-shard bias; no AllReduce (output shards concatenated) - RowParallelLinear: shards weight cols across TP ranks, forward_cpu() partial sum + bias on rank 0 only; async forward() calls ProcessGroup AllReduce for real NCCL path; CPU sim is no-op - TensorParallel::matmul() replaced zeros stub with ColParallelLinear(tp=1) - col→row roundtrip verified within 1e-3; 9 tests Co-Authored-By: Claude Sonnet 4.6 <[email protected]>	2026-06-27 03:54:07 +00:00
Omar SobhandClaude Sonnet 4.6	a670045f46	feat(perf): GPU perf batch 2 — EAGLE-3 dynamic draft trees + GaLore-2 optimizer ... EAGLE-3 dynamic draft trees (rtx-inference) - `eagle3.rs`: `Eagle3Config` (min_depth=1, max_depth=6, expansion_threshold=0.4, beam_width=3, self_consistency=true, prune_threshold=0.05), `DynamicDraftTree` with confidence-gated BFS expansion + iterative bottom-up cascade pruning + `all_paths()` / `accept_path()`, `Eagle3Decoder::build_draft_tree()` with cheap hidden-state proxy for child nodes (parent states scaled by child probability) - `tree.rs`: added `path_probability(leaf)`, `leaves()` (tombstone-safe DFS) - `types.rs`: added `DraftModelType::Eagle3` variant - 10 new unit tests via `FixedProbDraftModel` mock (no GPU required); total 85 pass GaLore-2 low-rank optimizer state (rtx-transformers) - `galore.rs`: `GaLoreConfig` (rank=128, update_proj_gap=200, scale=0.25, min_param_size=4096, momentum_inheritance=true), `GaLoreParamState` (proj_matrix [rows×rank], m_lr/v_lr [rank×cols]), `GaLoreAdamW` implementing `Optimizer` trait - Randomized range-finder sketched SVD: Ω~N(0,1) via LCG, Y=G@Ω, Gram-Schmidt QR - Momentum inheritance: project old m_lr onto new subspace on refresh - Automatic fallback to standard AdamW for params smaller than `min_param_size` - Memory ratio at rank=64, param=256×256: 2×(64×256) vs 2×(256²) = 25% of full state - `mod.rs`: `pub mod galore` + re-exports - 12 unit tests (all CPU); total 102+12 pass Combined: 85 + 114 = 199 lib tests pass across rtx-inference and rtx-transformers Co-Authored-By: Claude Sonnet 4.6 <[email protected]>	2026-06-26 22:43:09 +00:00
redclawsystems	4d88dc0584	Initial commit	2026-03-04 00:08:42 +00:00