Live Anthropic validation — and the platform-breaking bug it caught
Running the opt-in live suite (CM_LIVE_LLM=1 + ANTHROPIC_API_KEY) against
the real API immediately surfaced a launch blocker: Anthropic (and
OpenAI) restrict tool names to ^[a-zA-Z0-9_-]{1,128}$ — our ENTIRE
registry uses dotted names (clock.now, email.send, shell.exec, ...).
The scripted provider never enforced the pattern, so every real-model
deployment would have 400'd on the first tool call.
- Fix at the provider boundary, where it belongs: wire_tool_name /
internal_tool_name codec (dots <-> __) applied in BOTH HTTP providers
at all three sites (tools list, assistant tool_use echo, inbound
tool_use decode). Internal naming (DB step rows, scenarios, UI traces)
unchanged. Offline unit test round-trips every registry name through
the wire pattern
- New live tests, all passing against api.anthropic.com (Haiku 4.5):
- provider tool ROUND TRIP: real ToolUse arrives, ToolResult ships
back exactly as a checkpoint would reassemble it, model completes,
real usage events on the wire
- full runtime loop: real model calls clock.now, run completes, REAL
token usage metered, credits decremented
- the #1-risk validation: a real model's email.send intercepted ->
suspended -> approved -> checkpoint RESUMED against the live API ->
completed -> outbox exactly 1 (checkpoint/resume fidelity end to end)
- Stray TC_OPENAI_COMPAT_* envs renamed to CM_OPENAI_COMPAT_*
No credentials stored anywhere; the key was passed via env only.
163 Rust tests (+5 live, key-gated).
Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Fable 5
parent
add4f79fed
commit
5407111a89
@@ -44,7 +44,8 @@ impl AnthropicProvider {
|
||||
.map(|part| match part {
|
||||
ContentPart::Text { text } => json!({"type": "text", "text": text}),
|
||||
ContentPart::ToolUse { id, name, input } => {
|
||||
json!({"type": "tool_use", "id": id, "name": name, "input": input})
|
||||
json!({"type": "tool_use", "id": id,
|
||||
"name": crate::wire_tool_name(name), "input": input})
|
||||
}
|
||||
ContentPart::ToolResult {
|
||||
tool_use_id,
|
||||
@@ -78,7 +79,7 @@ impl LlmProvider for AnthropicProvider {
|
||||
.iter()
|
||||
.map(|t| {
|
||||
json!({
|
||||
"name": t.name,
|
||||
"name": crate::wire_tool_name(&t.name),
|
||||
"description": t.description,
|
||||
"input_schema": t.input_schema,
|
||||
})
|
||||
@@ -124,7 +125,9 @@ impl LlmProvider for AnthropicProvider {
|
||||
if block["type"] == "tool_use" {
|
||||
pending_tool = Some((
|
||||
block["id"].as_str().unwrap_or_default().to_owned(),
|
||||
block["name"].as_str().unwrap_or_default().to_owned(),
|
||||
crate::internal_tool_name(
|
||||
block["name"].as_str().unwrap_or_default(),
|
||||
),
|
||||
String::new(),
|
||||
));
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user