10 Commits

Include renames

Author	SHA1	Message	Date
osobhandClaude Sonnet 5	4aaa36a57a	style: cargo fmt --workspace (whitespace/wrapping only, no semantic change) ... Whole-workspace rustfmt pass picked up while iterating on Mamba GPU backward work. Verified formatting-only via diff sampling; no logic changed. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-08-10 07:09:36 -07:00
osobhandClaude Fable 5	733b02cd8b	feat(inference): concrete EAGLE draft model + real tokenizer at the serving boundary ... EAGLE (rtx-inference/src/eagle.rs, ~610 lines, mirrors medusa.rs conventions): EagleDraftHead autoregressive FFN with Concat/Add/ Attention feature fusion, EagleHeads draft model with draft/ draft_steps (per-step top-k for candidate trees) and teacher-forced training_loss; implements the speculative::EagleDraftModel trait so it plugs into the orchestration layer. 38 unit tests. Tokenizer (rtx-inference/src/tokenizer.rs): ServingTokenizer enum — Vocab (HuggingFace tokenizers, loadable from tokenizer.json) or ByteLevel fallback preserving previous behavior. rtx-serving-api's AppState and rtx-streaming's token generator now encode/decode through it (with_engine_and_tokenizer / set_tokenizer added; existing signatures unchanged). Also fixes two pre-existing compile errors in rtx-streaming (missing import, stray .await) that blocked its lib tests entirely. Tests: rtx-inference 328 pass, rtx-serving-api 193 pass, rtx-streaming 53 pass (2 pre-existing mock-server connection failures unrelated to these changes). Co-Authored-By: Claude Fable 5 <[email protected]>	2026-07-09 21:54:05 -07:00
osobhandClaude Fable 5	0cbfc1a739	fix(tests): repair rtx-onnx-codegen build and all pre-existing test failures in rtx-serving-api and rtx-runtime ... - rtx-onnx-codegen: re-export AttributeValue from ir (private-module import broke the whole crate; remaining errors were knock-ons). - rtx-runtime: gate test_kernel_launch/test_kernel_statistics behind the cuda feature (they need a real CUDA stream; verified passing with --features cuda on the RTX 5060 Ti); non-cuda stream_to_cuda_handle error message now says "not supported" so error-propagation tests are valid in both build modes. - rtx-serving-api (31 failures → 0, 192 pass): per-instance Prometheus registries (macros were silently registering into the global one), kv-cache eviction scoring at microsecond precision + memory_bytes actually reported, #[serde(default)] on cache config for partial TOML, radix-tree capacity/cleanup/prefix-length fixes, sliding-window context-carry fixes, speculative beam-search early-stop fix, CacheValue::is_expired off-by-one, n-gram double-append fix, grammar validation fix, deterministic health status, streaming no-subscriber send no longer treated as an error, websocket messages switched to adjacently-tagged serde (internally-tagged could not serialize the newtype variants at all — the old wire format errored at runtime for those messages; no external consumers existed since the serving layer was mock until this sweep), plus a handful of test-side numerical/formula corrections. Co-Authored-By: Claude Fable 5 <[email protected]>	2026-07-09 19:49:01 -07:00
osobhandClaude Fable 5	64ade03ab9	fix(production): wire real inference path through engine, serving, and streaming ... - rtx-inference: sample_next_token now copies the actual logits from the forward pass (Tensor::to_vec, last-token slice) instead of sampling from a fabricated all-zero vector; request metrics report measured queue/processing times instead of hardcoded constants. - rtx-serving-api: depends on rtx-inference; /v1/completions dispatches to a shared InferenceEngine (byte-level tokenization until a real tokenizer is threaded through) and returns 503 when no engine is loaded instead of mock text. ServingServer::with_engine attaches one. - rtx-streaming: depends on rtx-inference; generate_tokens delegates to an attached backend engine and errors without one instead of emitting "token_N" placeholders; tokenization is byte-level, not position-mod. - speculative decoding: document the orchestration (speculative/) vs implementation (medusa.rs/lookahead.rs) layering; CLAUDE.md no longer claims a standalone rtx-speculative-decoding crate. Co-Authored-By: Claude Fable 5 <[email protected]>	2026-07-09 19:05:27 -07:00
Omar SobhandClaude Sonnet 4.6	ef786c0ab1	feat(batch5): mid-batch injection, PagedAttn v2 defrag, fused RoPE kernel ... Continuous batching (rtx-serving-api): - ContinuousBatchingConfig: enable_mid_batch_injection (default true), injection_check_interval (default 1), max_injections_per_step (default 4) - ContinuousBatchingController: inject_into_active_batch() + try_inject_pending() allow new sequences to join a running decode batch after each step - BatchingError::BatchFull variant; 3 new tests PagedAttention v2 defrag (rtx-memory): - PageTable::fragmentation_ratio() — hole-counting (sandwiched free pages / total) - PageTable::defragment() — in-place left-compaction of physical page metadata, consistent lock order (free_pages -> physical_pages -> sequences); GPU KV copy stub comment; DefragStats return value; re-exported from lib.rs - 4 defrag tests; fixed 2 pre-existing compile errors in gpu_oom.rs + gpu_transfer.rs - 192 tests pass Fused RoPE kernel (rtx-transformers): - build_cos_sin_table() + rope_forward_cpu() CPU reference (norm-preserving) - RopeFusedKernel wrapper; rope_forward.cu CUDA kernel (1 block per (B,H,T), 1 thread per dim pair, NVRTC compiled) - Replaced apply_rope_rotation() mul_scalar(0.99) stub with real pairwise rotation - build.rs for NVRTC kernel tracking; layers/mod.rs wired; 8 tests pass Co-Authored-By: Claude Sonnet 4.6 <[email protected]>	2026-06-27 01:40:47 +00:00
Omar SobhandClaude Sonnet 4.6	1eb89c5b2b	feat(perf): GPU perf batch 1 — FP8, FA3 Blackwell, CUDA Graphs, SnapKV, prefix cache ... Item 1 — CUDA Graphs wiring (rtx-transformers) - Added `enable_cuda_graphs: bool` (default false) + `cuda_graph_warmup_iters: usize` (default 3) to `TrainingConfig` - Wired 3-phase state machine into `training_loop.rs` (warmup → capture → replay) gated on `#[cfg(feature = "cuda")]`; stream plumbing stubbed with TODO pending `CudaStreamHandle` threading Item 2 — FP8 E4M3/E5M2 training infrastructure (rtx-tensor, rtx-transformers) - `fp8_cast.cu`: dual-path CUDA kernels — SM_89+ uses `<cuda_fp8.h>` native `__nv_cvt_*` intrinsics; older SM uses software bit-manipulation fallback - `fp8_cast.rs`: host-side CPU casting + `#[cfg(feature = "cuda")]` GPU stubs - `fp8_gemm.rs`: bit-accurate E4M3 decoder/encoder, BF16 round-trip utils, CPU reference matmul with cuBLASLt GPU path documented inline; 12 unit tests - `training_config.rs`: `fp8_training: bool`, `fp8_e4m3_forward: bool` - `linear.rs` (modular): `fp8_mode: bool` field + forward dispatch stub - build.rs: registers `fp8_cast.cu` alongside existing `element_wise.cu` - 22 FP8 unit tests — all pass Item 3 — FlashAttention-3 Blackwell (WGMMA + TMA + warp specialization) - `flash_attention_v3_forward.cu`: SM_90+ warp-specialised producer/consumer kernel (producer TMA-loads K/V tiles, consumers run WMMA as portable WGMMA proxy); SM_89+ FP8 header path; SM<90 standard FA2-style WMMA fallback - `flash_v3_forward.rs`: NVRTC wrapper (`compile_ptx` via `include_str!`), `FlashV3ForwardKernel::new/is_supported/forward`; 6 unit tests - `backend_selector.rs`: `SdpaBackend::FlashAttentionV3`, `for_compute_capability`, `supports_flash_v3` (major >= 9), FA3 scoring (0.98/0.90/0.70), 2× speedup estimate - `kernels/simple.rs`: `v3_kernel: Option<FlashV3ForwardKernel>` in `FlashCudaKernels` - Fixed pre-existing `Device::Cpu` cfg-gate bug in `tensor/creation.rs` - 8 new FA3 backend tests + 6 kernel unit tests; 50 total pass Item 4 — SnapKV attention-score eviction + prefix caching (rtx-inference, rtx-serving-api) - `prefix_index.rs`: `PrefixIndex` with 8MB Zobrist hash table (Knuth MMIX LCG seed), `compute_hash/lookup/insert/remove/remove_page`; 10 unit tests - `eviction.rs`: `AttentionScoreEviction` struct — `accumulate_scores` + `select_evict_positions` (retain top keep_ratio + last recent_window); 7 unit tests - `types.rs`: `EvictionPolicy::AttentionScore { keep_ratio, recent_window }` + `KvCacheConfig::enable_prefix_caching` - `paged_kv_cache.rs`: `prefix_index: Option<PrefixIndex>` + `lookup_prefix / register_prefix / unregister_prefix_page / prefix_caching_enabled` methods - `config.rs` (serving-api): `enable_prefix_sharing: true` (was false), `snapkv_keep_ratio: 0.6`, `snapkv_recent_window: 32` - Fixed 12 pre-existing test errors (spurious `.await` on sync constructors) - 17 SnapKV/prefix tests pass Total: 918 lib tests pass across rtx-tensor, rtx-flash-attention, rtx-transformers, rtx-inference. Zero new failures. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>	2026-06-26 19:56:39 +00:00
Omar Sobh	16161bb9df	deps: align all 56 per-crate Cargo.toml files to thiserror v2 ... The workspace root was upgraded to thiserror = "2" in an earlier commit, but 56 per-crate Cargo.toml files still independently declared "1.0". These crates do not use workspace.dependencies inheritance for thiserror. All updated to thiserror = "2" for complete fleet alignment. Includes: rtx-backend, rtx-tensor, rtx-losses, rtx-backend-cuda/rocm/metal, all training crates (rtx-auto, rtx-rl, rtx-distributed, rtx-federated, etc.), specialized crates (rtx-science, rtx-platform, rtx-nmf, rtx-neuro-*), production crates (rtx-streaming, rtx-serving-api), and all demo crates. cargo check --workspace: PASSES.	2026-04-26 11:45:14 -07:00
osobhandClaude Opus 4.6	5fe8c67b04	Fix rtx-serving-api binary: create main.rs entry point ... The binary path pointed to non-existent src/bin/server.rs. Created proper src/main.rs with tokio async entry point that: - Initializes tracing - Loads config from RTX_HOST/RTX_PORT/RTX_TIMEOUT env vars - Instantiates ServingServer and calls serve() Fixed Cargo.toml: path = "src/bin/server.rs" → path = "src/main.rs" Validated: binary builds, starts, health endpoint returns healthy, inference endpoint returns mock completions. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>	2026-04-12 20:16:38 -07:00
osobhandClaude Opus 4.6	02d382d5f6	style: apply rustfmt across all crates and demos ... Consistent formatting pass: line wrapping, import sorting, trailing whitespace removal, let-chain indentation, merged derive attributes, and unsafe block reformatting. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>	2026-04-12 07:01:58 -07:00
redclawsystems	4d88dc0584	Initial commit	2026-03-04 00:08:42 +00:00