Generic LLM client abstraction for the conversational stack:
- LlmClient trait with generate_stream(messages, config) -> TokenStream.
Default generate() impl folds the stream for non-streaming callers.
- ChatMessage / Role / GenConfig types with sensible defaults.
- OpenAiCompatibleClient: HTTP impl with SSE streaming. Works against
OpenAI, Z.AI, vLLM, llama.cpp's HTTP server, LiteLLM — any endpoint
serving the Chat Completions schema.
- examples/llm_chat: demo CLI that prints token-by-token to stdout
with TTFT + total-time + char-count metrics.
Promotes tokio + reqwest + futures-util to regular dependencies (no
longer dev-only) so the trait is part of the public library surface.
Adds async-trait + eventsource-stream for the SSE streaming.
3 unit tests (constructors, role serialization, default config); 82
lib tests total green.
Phase 6 progress:
- 6a Kyutai STT: integration scaffolded; output bridge needs Python
reference diff (deferred)
- 6b LLM client: shipped (this commit)
- 6c session glue (axum WS + duplex audio loop): next
- 6d productionization: deferred
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
