perf(judge): a command may return 64 KB, so a deliverable is read once
7 of 9 verdicts ran to the 12-check cap. The checks say why: on a research mission, 8 of 12 commands read research/REPORT.md — cat, then head -119, tail -120, sed -n 80,200p and three greps. `cat` had come back truncated at 11,983 bytes because the per-command cap was 12 KB and the report was ~18 KB, so the judge reassembled the file in slices. Five extra rounds, each resending the whole conversation, to read one deliverable. The microVM verdicts used 5–7 checks because MICROVM.md is two lines. The cap was sized for test-suite output and applied to deliverables. 64 KB now. With compact_earlier_results shrinking a result to 800 bytes after its round, one 64 KB read costs one round; the slicing it replaces cost five. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
This commit is contained in:
co-authored by
Claude Opus 5
parent
2109bae6c6
commit
4e342d1ff7
@@ -50,7 +50,16 @@ const COMMAND_TIMEOUT: Duration = Duration::from_secs(180);
|
|||||||
/// Cap on what one command may return to the model. Test suites are chatty and
|
/// Cap on what one command may return to the model. Test suites are chatty and
|
||||||
/// the judge pays for every byte; the tail is where failures live, so when
|
/// the judge pays for every byte; the tail is where failures live, so when
|
||||||
/// output overflows we keep both ends and drop the middle.
|
/// output overflows we keep both ends and drop the middle.
|
||||||
const MAX_OUTPUT_BYTES: usize = 12_000;
|
///
|
||||||
|
/// 64 KB, up from 12 KB on 2026-09-18. The smaller cap was sized for test
|
||||||
|
/// output and applied to deliverables: a research REPORT.md of ~18 KB came
|
||||||
|
/// back truncated from `cat`, and the judge — correctly — reassembled it with
|
||||||
|
/// `head -119`, `tail -120`, `sed -n 80,200p` and three greps, five extra
|
||||||
|
/// rounds each resending the whole conversation. 7 of 9 verdicts ran to the
|
||||||
|
/// 12-check cap that way. Since `compact_earlier_results` shrinks a result to
|
||||||
|
/// 800 bytes once its round is over, one 64 KB read costs one round; the
|
||||||
|
/// slicing it replaces cost five.
|
||||||
|
const MAX_OUTPUT_BYTES: usize = 64_000;
|
||||||
|
|
||||||
/// Programs the judge may run. Every one either reports state or runs a
|
/// Programs the judge may run. Every one either reports state or runs a
|
||||||
/// project's own checks — none of them edit the tree.
|
/// project's own checks — none of them edit the tree.
|
||||||
|
|||||||
Reference in New Issue
Block a user