Omar SobhandClaude Opus 5
4358964c05
55 of 85 role skill bindings pointed at skills that were never authored,
so 10 of 11 team templates bound a smaller context bundle than their role
prompts assumed. Three roles bound nothing at all (gpu.bench_engineer,
threejs.shader_author, threejs.perf_engineer) while their prompts described
procedures they had no way to read.
The loader comment at team_template_loader.rs:167 already diagnosed this —
snake_case slugs in TOML against kebab-case skill files — and it was
half-fixed: the kebab names were corrected, the snake_case ones left.
It was invisible because both existing tests assert authored ⊆ referenced
(30/30, green) and the second explicitly declines to check the other
direction. So the failing half was the half nobody asserted.
Resolved every name by one of three explicit choices:
- 23 skills authored where the role genuinely needed the procedure
(gpu, threejs, research, analysis, frontend, mobile, backend, platform)
- renames onto authored skills where one existed in substance, including
the four-near-duplicate cases that collapse onto one real skill
- 22 aspirational references deleted — a binding an agent cannot read is
a promise, not a capability
Two tests now hold it. The unit test checks referenced ⊆ authored against
the files. The new integration test runs both loaders in boot order and
asserts the bindings survive the trip through the database, which is a
different question: resolution goes through skills_catalog rows, so a skill
file that exists but fails to ingest still leaves the role empty.
Negative controls: the unit test failed naming all 55; the integration test
fails naming the exact role when one name is reverted.
threejs.shader_author and .perf_engineer gained a second and third skill
after the collapse — pin_in_context pins idx < 2, so a role left with one
skill silently pins less than the policy intends.
Co-Authored-By: Claude Opus 5 <[email protected]>
2.2 KiB
name, description, when_to_use, tags
| name | description | when_to_use | tags | ||
|---|---|---|---|---|---|
| level-up-proposal-shape | Writing an agent-improvement proposal that a human can accept or reject on the evidence, not on the prose. | You are proposing a change to an agent's definition, skills or model. |
|
A proposal is an argument with evidence attached
Level-up proposals go to a human review gate. The gate is the safety mechanism, so a proposal that cannot be judged quickly either gets rubber-stamped or ignored — both failures.
The shape
OBSERVATION what the agent did, with a run or event to point at
DIAGNOSIS which part of the definition caused it
CHANGE the exact edit — the new text, not a description of it
EXPECTED what will differ next time, observably
RISK what this could make worse
EXPECTED is what makes the proposal checkable later. "Better research quality"
cannot be verified; "will cite the file it changed, as paper-to-project-relevance
requires" can.
Point at evidence that will still exist
mission_events is retained for seven days. A proposal citing an event
older than that points at nothing by the time anyone re-reads it. Quote the
relevant text inline rather than referencing an id alone.
Prefer the smallest edit that could work
A rewritten system prompt is unreviewable — a human cannot tell which of forty changed lines caused the next difference in behaviour. One change per proposal, so the outcome attributes to something.
Say which layer
The same symptom has different fixes at different layers, and naming the wrong one wastes a review cycle:
- System prompt — the agent's standing identity and constraints.
- Skill — a procedure it should follow when a situation arises.
- Model — capability, when the agent understood the task and could not do it.
- Nothing — the task was ambiguous and the agent behaved reasonably.
"Nothing" is a legitimate proposal outcome and should be written when it is true.
Reject-worthy proposals
If you cannot name the observation, or the change is "be more careful", it is not a proposal. Vague instructions do not change behaviour, and they accumulate in the prompt where they crowd out the specific ones that do.