feat(inference): concrete EAGLE draft model + real tokenizer at the serving boundary

EAGLE (rtx-inference/src/eagle.rs, ~610 lines, mirrors medusa.rs
conventions): EagleDraftHead autoregressive FFN with Concat/Add/
Attention feature fusion, EagleHeads draft model with draft/
draft_steps (per-step top-k for candidate trees) and teacher-forced
training_loss; implements the speculative::EagleDraftModel trait so it
plugs into the orchestration layer. 38 unit tests.

Tokenizer (rtx-inference/src/tokenizer.rs): ServingTokenizer enum —
Vocab (HuggingFace tokenizers, loadable from tokenizer.json) or
ByteLevel fallback preserving previous behavior. rtx-serving-api's
AppState and rtx-streaming's token generator now encode/decode through
it (with_engine_and_tokenizer / set_tokenizer added; existing
signatures unchanged). Also fixes two pre-existing compile errors in
rtx-streaming (missing import, stray .await) that blocked its lib
tests entirely.

Tests: rtx-inference 328 pass, rtx-serving-api 193 pass, rtx-streaming
53 pass (2 pre-existing mock-server connection failures unrelated to
these changes).

Co-Authored-By: Claude Fable 5 <[email protected]>

crates/production/rtx-serving-api/src/server.rs

@@ -47,16 +47,33 @@ impl ServingServer {
         }
     }
 
-    /// Create a serving server backed by a live inference engine
+    /// Create a serving server backed by a live inference engine.
+    ///
+    /// Uses byte-level tokenization by default; use
+    /// [`ServingServer::with_engine_and_tokenizer`] to attach a real
+    /// vocabulary tokenizer.
     #[must_use]
     pub fn with_engine(
         config: ServerConfig,
         engine: std::sync::Arc<tokio::sync::RwLock<rtx_inference::InferenceEngine>>,
+    ) -> Self {
+        Self::with_engine_and_tokenizer(config, engine, rtx_inference::ServingTokenizer::default())
+    }
+
+    /// Create a serving server backed by a live inference engine and an
+    /// explicit tokenizer (e.g. loaded via
+    /// `rtx_inference::ServingTokenizer::from_file`).
+    #[must_use]
+    pub fn with_engine_and_tokenizer(
+        config: ServerConfig,
+        engine: std::sync::Arc<tokio::sync::RwLock<rtx_inference::InferenceEngine>>,
+        tokenizer: rtx_inference::ServingTokenizer,
     ) -> Self {
         Self {
             config,
             state: inference::AppState {
                 engine: Some(engine),
+                tokenizer,
             },
         }
     }