osobhandClaude Opus 4.7
a5cedfb46a
8-gen bench (4 emotions × 2 corpora) at seed=42 against firdhokk Whisper-LV3: target RAVDESS CREMA-D happy happy (0.999) ✓ happy (0.999) ✓ angry neutral (0.92) sad (0.99) fearful happy (0.998) fearful (0.984) ✓ sad angry (0.99) fearful (0.99) CREMA-D 2/4 vs RAVDESS 1/4. Larger / more naturalistic corpus produces more class-pure fearful direction. Neither corpus solves angry or sad — recipe shifts into 'vague expressivity' rather than class-specific corners. Practical: prefer CREMA-D when available; A/B both per emotion if class precision matters. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
55 lines
2.0 KiB
Rust
55 lines
2.0 KiB
Rust
//! Smoke test for `GenConfig::extra_body` plumbing. Calls Z.AI directly
|
|
//! with `{"thinking":{"type":"disabled"}}` and times the streaming round-
|
|
//! trip. If extra_body is properly merged, latency should be ~1s; if the
|
|
//! field is dropped, the model burns tokens on reasoning_content first
|
|
//! and latency balloons to 30+ seconds.
|
|
|
|
use anyhow::Result;
|
|
use futures_util::StreamExt;
|
|
use rtx_csm::llm_client::{ChatMessage, GenConfig, LlmClient, OpenAiCompatibleClient};
|
|
use std::time::Instant;
|
|
|
|
#[tokio::main]
|
|
async fn main() -> Result<()> {
|
|
let api_key = std::env::var("ZAI_API_KEY")
|
|
.unwrap_or_else(|_| "0ca491ae08594e1e98fe6d4061990d6b.nGcDsViIIsS7xuzf".into());
|
|
let client =
|
|
OpenAiCompatibleClient::new("https://api.z.ai/api/coding/paas/v4", api_key, "glm-4.5");
|
|
|
|
for label in ["WITHOUT extra_body", "WITH thinking-disabled"] {
|
|
let mut cfg = GenConfig {
|
|
max_tokens: Some(160),
|
|
temperature: 0.7,
|
|
..GenConfig::default()
|
|
};
|
|
if label.contains("WITH") {
|
|
let mut m = serde_json::Map::new();
|
|
m.insert("thinking".into(), serde_json::json!({"type":"disabled"}));
|
|
cfg.extra_body = m;
|
|
}
|
|
let messages = vec![
|
|
ChatMessage::system("Reply in one short sentence."),
|
|
ChatMessage::user("Describe stew."),
|
|
];
|
|
let t = Instant::now();
|
|
let mut stream = client.generate_stream(messages, cfg).await?;
|
|
let mut first_chunk_ms: Option<u128> = None;
|
|
let mut full = String::new();
|
|
while let Some(c) = stream.next().await {
|
|
let s = c?;
|
|
if first_chunk_ms.is_none() {
|
|
first_chunk_ms = Some(t.elapsed().as_millis());
|
|
}
|
|
full.push_str(&s);
|
|
}
|
|
println!(
|
|
"{label}: ttf_chunk={}ms, total={}ms, len={} chars",
|
|
first_chunk_ms.unwrap_or(0),
|
|
t.elapsed().as_millis(),
|
|
full.chars().count()
|
|
);
|
|
println!(" reply: {}", full.trim());
|
|
}
|
|
Ok(())
|
|
}
|