Whole-workspace rustfmt pass picked up while iterating on Mamba GPU
backward work. Verified formatting-only via diff sampling; no logic
changed.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
The new lifecycle tests caught that the batch-optimizer worker crashed at
spawn: DynamicBatchingConfig derived Default (all zeros), and tokio's
interval() panics on a zero period. Default is now a usable config
(batch 32 in [1,128], step 4, 1ms latency target, 100-sample window,
1s optimization interval, AIMD adaptation).
New regression tests in AdaptiveProcessor, EdgeComputingManager, and
MonitoringSystem assert that all workers are still alive shortly after
start() (catches workers dying at startup) and that stop() completes via
the graceful control-channel path, not the 5s abort backstop (catches
shutdown hangs).
cargo test -p rtx-streaming: 58 lib + 8 integration + 6 aux, all green.
Co-Authored-By: Claude Fable 5 <[email protected]>
Follow-up to a0bf294, which tolerated dead control channels; this makes
them functional:
- AdaptiveProcessor: mpsc control channel (single consumer behind a
mutex, broke on ANY message including Start) replaced with broadcast;
all three workers (resource monitor, batch optimizer, pressure
monitor) subscribe and exit only on ControlCommand::Stop
- EdgeComputingManager / MonitoringSystem: their 7 interval-loop workers
now subscribe to the existing broadcast control channels and exit on
Stop instead of looping forever
- stop() in all three: graceful join with 5s timeout, abort only as a
backstop (previously unconditional abort mid-tick)
- benches: criterion needs async_tokio for Bencher::to_async — bench
target now compiles (clippy --all-targets clean)
cargo test -p rtx-streaming: 55 lib + 8 integration + 6 aux, all green.
Co-Authored-By: Claude Fable 5 <[email protected]>
- token_generator: backend is now an optional real rtx-inference engine
(RwLock<Option<Arc<InferenceEngine>>>) with ServingTokenizer support;
set_backend/set_tokenizer plumbing through StreamingServer
- connection_manager: ConnectionPool::acquire no longer errors when the
idle cache is full — creates fresh connections up to max_connections
- streaming_server: ServerState::Running on construction; stream_inference
generates one token per step (chunk_size semantics)
- lifecycle bugs surfaced by the newly-compiling integration tests:
* start(): broadcast control-channel send with zero subscribers was
treated as fatal ("channel closed") in RealtimePipeline,
EdgeComputingManager, MonitoringSystem — now tolerated
* stop(): AdaptiveProcessor/EdgeComputingManager/MonitoringSystem
awaited worker interval loops that never exit (test hung 5h) —
workers are now aborted with cancellation-aware join
- integration_tests: removed stale .await on now-synchronous methods
cargo test -p rtx-streaming: 55 lib + 8 integration + 6 aux, all passing.
Co-Authored-By: Claude Fable 5 <[email protected]>
EAGLE (rtx-inference/src/eagle.rs, ~610 lines, mirrors medusa.rs
conventions): EagleDraftHead autoregressive FFN with Concat/Add/
Attention feature fusion, EagleHeads draft model with draft/
draft_steps (per-step top-k for candidate trees) and teacher-forced
training_loss; implements the speculative::EagleDraftModel trait so it
plugs into the orchestration layer. 38 unit tests.
Tokenizer (rtx-inference/src/tokenizer.rs): ServingTokenizer enum —
Vocab (HuggingFace tokenizers, loadable from tokenizer.json) or
ByteLevel fallback preserving previous behavior. rtx-serving-api's
AppState and rtx-streaming's token generator now encode/decode through
it (with_engine_and_tokenizer / set_tokenizer added; existing
signatures unchanged). Also fixes two pre-existing compile errors in
rtx-streaming (missing import, stray .await) that blocked its lib
tests entirely.
Tests: rtx-inference 328 pass, rtx-serving-api 193 pass, rtx-streaming
53 pass (2 pre-existing mock-server connection failures unrelated to
these changes).
Co-Authored-By: Claude Fable 5 <[email protected]>
- rtx-inference: sample_next_token now copies the actual logits from the
forward pass (Tensor::to_vec, last-token slice) instead of sampling
from a fabricated all-zero vector; request metrics report measured
queue/processing times instead of hardcoded constants.
- rtx-serving-api: depends on rtx-inference; /v1/completions dispatches
to a shared InferenceEngine (byte-level tokenization until a real
tokenizer is threaded through) and returns 503 when no engine is
loaded instead of mock text. ServingServer::with_engine attaches one.
- rtx-streaming: depends on rtx-inference; generate_tokens delegates to
an attached backend engine and errors without one instead of emitting
"token_N" placeholders; tokenization is byte-level, not position-mod.
- speculative decoding: document the orchestration (speculative/) vs
implementation (medusa.rs/lookahead.rs) layering; CLAUDE.md no longer
claims a standalone rtx-speculative-decoding crate.
Co-Authored-By: Claude Fable 5 <[email protected]>
