8-gen bench (4 emotions × 2 corpora) at seed=42 against firdhokk
Whisper-LV3:
target RAVDESS CREMA-D
happy happy (0.999) ✓ happy (0.999) ✓
angry neutral (0.92) sad (0.99)
fearful happy (0.998) fearful (0.984) ✓
sad angry (0.99) fearful (0.99)
CREMA-D 2/4 vs RAVDESS 1/4. Larger / more naturalistic corpus
produces more class-pure fearful direction. Neither corpus solves
angry or sad — recipe shifts into 'vague expressivity' rather than
class-specific corners.
Practical: prefer CREMA-D when available; A/B both per emotion if
class precision matters.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Adds `extra_body: serde_json::Map<String, Value>` to `GenConfig`. The
OpenAiCompatibleClient serializes the typed ChatRequest to a Value, then
merges extra_body's keys at the top level of the request body before
sending. extra_body is empty by default, so existing callers see no
behavioral change.
Use cases:
- **Z.AI thinking-disabled** for voice-AI: glm-4.5/4.6/4.7 default to
reasoning_content traversal which burns tokens before content emits.
Pass `{"thinking":{"type":"disabled"}}` and reasoning_tokens drops to
0. Verified: curl direct = 2.1s vs 30s+ with thinking.
- **vLLM guided decoding**: `{"guided_json": {...}}`.
- **Anthropic-compat thinking budget** (when proxied through an
OpenAI-compat shim).
Wires `--llm-extra-body '<JSON>'` into examples/converse_server: parsed
once at boot, stored in Shared.llm_extra_body, cloned per-turn into
gen_cfg. Boot rejects malformed JSON or non-object payloads.
Smoke test: examples/llm_extra_body_smoke.rs hits Z.AI directly with
and without extra_body, prints ttf_chunk and total stream time. Latest
run on glm-4.5: WITHOUT extra_body 1283ms, WITH thinking-disabled
1385ms — both fast on this prompt; the field is correctly forwarded
either way (other prompts that trigger reasoning_content show the
30s+ delta).
Note: the first end-to-end test through converse_server still showed
~46s wall (vs 1.4s for the LLM call alone), implying the latency
bottleneck is local STT (~12s on this hardware) + TTS gen, not the
LLM. extra_body code path is verified independently via the smoke
test.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Generic LLM client abstraction for the conversational stack:
- LlmClient trait with generate_stream(messages, config) -> TokenStream.
Default generate() impl folds the stream for non-streaming callers.
- ChatMessage / Role / GenConfig types with sensible defaults.
- OpenAiCompatibleClient: HTTP impl with SSE streaming. Works against
OpenAI, Z.AI, vLLM, llama.cpp's HTTP server, LiteLLM — any endpoint
serving the Chat Completions schema.
- examples/llm_chat: demo CLI that prints token-by-token to stdout
with TTFT + total-time + char-count metrics.
Promotes tokio + reqwest + futures-util to regular dependencies (no
longer dev-only) so the trait is part of the public library surface.
Adds async-trait + eventsource-stream for the SSE streaming.
3 unit tests (constructors, role serialization, default config); 82
lib tests total green.
Phase 6 progress:
- 6a Kyutai STT: integration scaffolded; output bridge needs Python
reference diff (deferred)
- 6b LLM client: shipped (this commit)
- 6c session glue (axum WS + duplex audio loop): next
- 6d productionization: deferred
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
redclawsystems