Judge spend gained provider, model and mission on 2026-09-14; agent spend —
the larger half — did not. The runtime's `done` frame has always carried
`model` and `provider` beside the two token counts, and `topology_exec` read
only the counts, summed them, and charged the sum as output with no record of
which provider served the turn.
`TurnOutcome` and `StepRecord` carry a `Spend` now (input/output split,
provider, model), the worker passes it through `cm_billing::charge` along with
the mission id, and the chat runtime records the model it requested — that
loop drives one provider with no chain, so requested is answered. A bare
model name is recorded without a guessed family. `StepRecord.spend` is
`serde(default)` so journaled checkpoints from before this field still load,
and `tokens` stays as the total every reader keys on.
`charge` moved from `query!` to `query`: the macro pins the statement to
offline metadata that a schema change then has to regenerate against a live
database, for columns that are nullable text and uuid.
The done-frame test now asserts the split and the provider survive, not just
the sum.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
Make Scorer async and add JudgeScorer (behind the `provider` feature): asks a
cm-llm model to rate a run's output 0-100 vs the task and normalizes to [0,1],
giving the comparison harness real quality numbers. Robust integer parsing
(handles "Score: 92/100", clamps >100); provider errors score 0.0.
11 tests with --features provider (judge incl. parse + scripted-provider score);
core stays 7. Clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
compare(graphs, task, executor, scorer) runs the same task across a set of
topologies on the same executor, scores each, and returns a Comparison:
- per-topology results (quality, tokens, turns, blocked approvals, output),
- a leaderboard (quality desc),
- a quality/cost Pareto front (on_pareto flags),
- best_quality and best_value (quality-per-token) picks.
This is the "which patterns yield better results" engine and the structured
output the paper's benchmark tables consume. Generic over TurnExecutor +
Scorer (pluggable LLM-judge later); pure core, no new deps. 9 tests, clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
