09c64967250b5e3408fa527f2765ddb35e1a15f0
15
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar SobhandClaude Opus 5
|
09c6496725 |
feat(library): clone the vault, harvest our topics, push the catalogue
Completes the loop: the notes now land in the real vault. Topics come from what the project is actually working on — papers/dynamic-agentic- topologies.md (topology search, ADAS/Darwin-Godel/SwarmAgentic) plus the two problems this week ran into, verifying what an agent did and giving a long-running agent memory of what it covered. Never pushes to main. The vault is a live Obsidian vault a human edits and syncs; pushing to main races that sync and can lose hand-written work. Every run lands on its own branch for a human to merge, the same rule the mission delivery path was validated 20/20 under. PDFs are NOT committed. A few hundred papers is gigabytes and would make the vault painful to clone and slow to open, so they stay on the blob store shelf and the note carries the key. My own test caught me repeating this week's branch-collision bug: I named branches from the HEAD of a UUIDv7, which is a 48-bit timestamp, so two runs in the same millisecond produce the identical name — exactly what hit mission 019fc42b. Fixed by taking the tail. The test now loops 100 ids instead of sampling two (a one-shot check passes by luck whenever the millisecond ticks between calls) and additionally asserts the head-based scheme DOES collide, so it cannot rot into a no-op. Live against the real vault: 10 candidates, 1 already held, 9 shelved, 0 failed branch clawmates/library-019fc82292e8, pushed 9 notes verified on the forge, 9 PDFs verified %PDF on the shelf (the "1 already held" is cross-topic dedupe inside a single run) Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
ec85f6c8da |
fix(missions): close the three seams behind this run of failures
Seam 1 — delivery inferred checkout state from the tree, so whether work survived depended on what the agent happened to do. 019fc444 committed and left a clean tree; 019fc476 had its base advanced to match HEAD; 019fc450 survived only because a phase FAILED to commit and left the tree dirty. Same code, opposite outcomes, decided by the agent. mark_phase_started records the fact at phase launch, before the agent acts, so every one of those states answers identically. The tree checks remain as a second line of defence for pre-existing checkouts. Seam 2 — phase config was accepted, stored and read by nobody. That was `task`: every phase of every mission got identical instructions. The new phase_config registry names the reader for each live key and lists the eight that are declared-but-unimplemented, reporting both at mission creation so an author sees what will not happen. Its CI test found one I had missed: security_hardening.toml sets phase-level mcp_bundles asking for gitea_forge + security_scan, but bundles come from the TEAM template and the phase gets neither. Seam 4 — push_url_for collapsed a failed query, an unbound repo and a missing clone_url into one None, so a database fault was recorded as "nothing to push to" and metadata read `pushed: null, push_error: null` — the same ambiguity commit_error already fixed. Each case now carries its reason into the artifact, and a local git failure during publish is recorded rather than dropped by .ok(). Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
1a979f500f |
fix(missions): judge local work against the remote tip, not the capture base
Mission 019fc476 lost phase 0's work again, and this time the cause was the interaction between two fixes I had just shipped. has_local_work compared HEAD against .git/clawmates-base to decide whether a checkout held mission work. advance_base_commit moves that marker to each phase's committed head. So the moment a phase committed successfully, base == HEAD, has_local_work reported "pristine", and the next phase's launch reset the work away. Phase 1 wrote CHAIN_MISSING.md. The preceding mission survived only because its phase 0 FAILED to commit and left a dirty tree. Fixing that failure is what exposed this one. One marker was carrying two meanings: "where should the next diff start" (rolling, per phase) and "is this checkout untouched" (fixed for the mission). Only the first belongs to clawmates-base. The second is now `origin/<branch>`, which does not move for the life of the mission, so a HEAD that differs from it means a phase committed — one commit ago or five. An unresolvable remote ref preserves, since wrongly skipping a refresh costs staleness while wrongly resetting destroys a phase. The existing test passed throughout because it never advanced the base. It now does, which makes it a reproduction rather than a restatement, and it needs a real bare origin to resolve origin/main the way a clone does. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
08b2adae23 |
fix(missions): stop resetting a checkout that holds mission work
Mission 019fc444 ran two coding phases. Phase 0 created ALPHA.md and
delivery committed it; phase 1 then started and ALPHA.md was gone from
the working tree, so the second phase never saw the first's output.
`ensure_checkout` is called at every phase launch, not once per mission,
and its reuse path runs `git reset --hard origin/<branch>`. That is right
for a checkout picked up cold and destructive for one mid-mission.
Delivery is what made this reachable. Before the mission branch existed,
agent output stayed untracked and a hard reset left it alone. Committing
it makes it tracked, and tracked files absent from origin/<branch> are
exactly what a hard reset removes — so the slice written to stop work
being destroyed is what put it in reach of the thing destroying it. The
flagship shape is the casualty: in research_and_code, the coding phase
never sees the research brief.
`has_local_work` now gates the refresh. It checks both a dirty tree and a
HEAD that has moved off the recorded base, because the two failure shapes
differ: an agent that committed leaves a CLEAN tree at a new HEAD, which
a dirty-tree check alone would miss — and that is precisely the shape
being destroyed. With no recorded base it preserves, since wrongly
skipping a refresh costs staleness while wrongly resetting costs a phase.
This also makes the base-advance fix in
|
||
|
Omar SobhandClaude Opus 5
|
5b53705c97 |
fix(missions): let the server and the agent share one git checkout
Mission 019fc437 lost both phases' work to:
git add → exit 128: insufficient permission for adding an object
to repository database .git/objects
cm-api runs as uid 65532; the mission runtime container runs as root;
they share one bind-mounted checkout. Git's .git/objects/xx/ fan-out
directories inherit the ownership of whoever creates them, so an agent
that writes objects first locks the server out of those directories.
The failure is intermittent, which is why the previous run looked clean.
Mission 019fc42b's agents committed their own work, so the blobs already
existed and the server's `git add` never had to write one. Same template,
different agent behaviour, opposite outcome.
`core.sharedRepository` is git's own mechanism for this: objects and refs
are created group- and world-writable, and both parties read the setting
from the shared .git/config. It grants the agent nothing — it is already
root over the whole checkout — and unblocks the server, which was the
party being refused. Applied on clone and on checkout reuse.
Two supporting changes. The artifact now records `commit_error`: this
failure surfaced as `branch: null, push_error: null`, indistinguishable
from a phase that never had work to commit, with the reason only in host
stderr. And the test seeder now calls the production setup function
instead of reimplementing it — building the checkout by hand is what let
a clone-path defect stay invisible to fourteen tests.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
8bad869248 |
fix(missions): give each phase its own task and its own capture base
The first push run against a scratch repo (mission 019fc42b) delivered two branches correctly but exposed two bugs behind them. Per-phase instructions were inert. `phase_task_text` took only (kind, title, description), so `mission_phases.config.task` was accepted by the API, stored, and read by nothing. Every phase of a mission received byte-identical text differing only by the kind directive — so both coding phases did the whole mission instead of their slice, producing the same two files. The task now reaches the agent as a trailing THIS PHASE'S TASK block, scoped against the shared brief. The capture base never advanced. `.git/clawmates-base` is written once at clone time, so phase two diffed against the original clone point and reported the union of both phases' files as its own. It now moves to each phase's committed head after the patch is on disk; the pushed branch stays cumulative because it is built from HEAD. Both regression tests were confirmed to fail with their fix disabled. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
e2871c4361 |
feat(missions): publish the mission branch, gated by commit_policy
Completes delivery. A phase's work is now captured, committed, gated and pushed — in that order, so every failure costs strictly less than the one before it. Publishing is last for a reason. By the time it runs the patch is on disk, the artifact is registered and the work is on a local branch, so a rejected ref, a rotated token or an unreachable forge costs a push and nothing else. A test pushes at a path that does not exist and asserts the commit is still there afterwards. The gate decides the branch name, never whether the work survives: - green, or policy `always` → `clawmates/mission-<m8>-<p8>` - red / unrunnable / no suite → `…-wip` - `on_reviewer_approval` → `…-review` Both land on the forge. A human can inspect, fix and re-push a branch; nobody can recover work discarded for failing a test. Deleting a red branch reproduces the old behaviour on purpose rather than by accident. `verify_tests` runs the project's own suite through the runtime container and returns `Option<bool>` — `None` for "could not establish", which the gate treats as unproven. An unreadable exit status is not a pass. That is the same fail-closed stance as the phase evaluator, and it is here because this tranche has now found four separate things reporting success while doing nothing. Never force-push. A rejected update is reported and left alone: the remote ref belongs to whoever set it, and overwriting it to make delivery look tidy is how a mission eats someone else's commit. The push URL is built fresh from the repo row and the ambient token, not read from `.git/config` — which no longer carries credentials, since agents run as root in a container that mounts the checkout. Tests push to a real `git init --bare` remote and assert the ref and its content actually arrived. A mock would have accepted anything. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
e089360ac8 |
fix(missions): unblock the capture batch, and restore fetch auth
Two defects, both found by running a second real coding mission (019fc3ba)
after the first round of fixes. The agent created the file correctly this
time — `file_write` did its job — and capture still produced nothing.
**Head-of-line blocking.** `capture_phase_diff` returns `Ok(None)` when the
checkout is gone, and the caller treated that as success without recording
anything. The phase therefore stayed eligible forever, and because the batch
is bounded at five, five reaped phases from earlier test missions occupied
every slot permanently. A freshly finished coding phase, with its checkout
still on disk, was never reached — and nothing was logged, because nothing had
failed.
Fixed on both axes: an unreachable checkout now writes a `code_diff` marker
recording `captured: false` and why, so the row stops being selected; and the
batch orders newest-first, so live work is captured before archaeology. The
marker also distinguishes "this phase changed nothing" from "we lost the
checkout before looking", which an operator reading the mission needs to be
able to tell apart.
**Fetch lost its credentials.** `scrub_remote_credentials` (P1.1) strips the
token from `.git/config` so agents running as root cannot read it — but
`fetch_and_reset` fetched from the stored remote, which is now anonymous:
git fetch origin <branch> → exit 128:
fatal: could not read Username for 'https://git.redclaw.dev'
I accounted for push building a fresh authenticated URL and overlooked that
fetch needs one too. `fetch_and_reset` now takes the authenticated URL the
caller already computes, as does the `--unshallow` deepen. Stderr stays
redacted.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
409ca65ee7 |
fix(missions): capture from the clone point, and let agents create files
Two defects found by running a real coding mission (019fc372) rather than a test. Both made a coding phase look like it produced nothing. **Capture measured the wrong baseline.** It diffed the working tree against HEAD, which is correct only while work stays uncommitted. `rust_sdlc` has a *committer* role, so committing is the intended path — meaning a mission that did its job properly leaves a clean tree and captured nothing. That is exactly what happened: the agent created `DELIVERY_PROBE.md`, committed it as `aa3be95`, and the artifact recorded `empty: true` beside a commit that plainly contained the work. `mission_workspace` now records the clone point in `.git/clawmates-base` (in `.git/`, so it travels with the checkout, stays invisible to the repository, and cannot be reached by an agent through its pinned workspace), refreshed whenever `fetch_and_reset` moves HEAD. Capture diffs from there, covering committed, staged and unstaged changes in one pass. Checkouts predating the marker fall back to HEAD and say so via `base_recorded: false`. **Agents could not create files.** `coding_readwrite` granted `file_edit` but not `file_write`. `file_edit` replaces an exact existing string and rejects an empty `old_string`, so creating a new file was impossible. The mission transcript is unambiguous: "the tool rejected empty old_string... the shell is restricted", after which the agent worked around it through `shell`. The comment above that profile has claimed it grants file_write since the day it was written; the list never contained it. Also broadens capture from coding/benchmark/security_scan to every phase kind of a repo-bearing mission: `phase_task_text` tells research phases to "save findings under /mission/repo/research/", so filtering by kind would have discarded every research brief such a mission produced. Regression tests cover committed-only and committed-plus-uncommitted work against a real git repo. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
716ee9a304 |
feat(missions): capture a coding phase's diff to durable storage
First half of mission delivery: the work is captured before anything is published. A coding mission has until now produced nothing durable — the checkout is deleted thirty minutes after completion and `register_artifact` had no callers at all, so the only surviving output was an LLM narrative of what the agents said they did. `capture_phase_diff` writes `diff.patch`, `diffstat.txt` and `delivery.json` under `<missions_root>/_outputs/<mission>/<phase>/` and registers a `code_diff` artifact. That directory is a *sibling* of the per-mission directories the sweeper removes, and outside every bind mount handed to a container — so teardown cannot take the record with it and agents cannot edit their own evidence. Three details that decide whether this works at all: - `git add --intent-to-add` before diffing. Untracked files are invisible to `git diff`, and a phase that only *creates* files is the likeliest shape for generated code — silently capturing an empty patch would be the worst possible failure. The index is reset afterwards so capture leaves the tree exactly as the agents left it, which the test asserts. - Build output is excluded by pathspec (`target`, `node_modules`, `.venv`, …). A phase that ran `cargo build` leaves a directory larger than the repo. - An empty diff is still an artifact, flagged `empty: true`. "This coding phase wrote no code" is currently invisible to an operator and is worth saying out loud. `RegisterArtifact` gains `metadata`, which the column has had since 0047 and nothing ever wrote; the diffstat and base sha go there. No migration needed — `kind` is unconstrained TEXT and the column already exists. Tests run against a real `git init` repo rather than a mock: every bug in this area so far came from git behaving differently than assumed, and a fake git would have agreed with the assumption. `capture_phase_diff_at` takes explicit paths so parallel tests cannot race through the process-global CLAWMATES_MISSIONS_ROOT — the first version of these tests did exactly that and two of four failed non-deterministically. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
ea3d145aac |
fix(missions): stop leaving an access token in every mission checkout
`with_ambient_auth` embeds GITEA_TOKEN in the clone URL, and git persists that
URL verbatim as the `origin` remote. The checkout is bind-mounted into a
container the agents run in as root, so the token sat in a file every mission
agent could read — and it reaches every repository that token reaches, not
just the one being worked on.
The remote is now rewritten to the bare URL immediately after clone. Delivery
does not depend on the stored URL: it will build a fresh authenticated URL at
push time, which also means a rotated token starts working at once rather than
after the next clone. Best-effort and non-fatal — a checkout that keeps its
token still works, and failing a mission over it would trade a real capability
for a situation already logged.
`strip_credentials` only treats an `@` in the *authority* as a separator, so a
path containing `@` (scoped npm-style names) is left alone.
Also, two changes delivery needs:
- `--depth 1` becomes `--filter=blob:none --single-branch`. A shallow clone
usually cannot push a new branch ("shallow update not allowed"), which is
exactly what mission delivery must do. A partial clone keeps full history —
so a base commit stays meaningful and a diff has something to be relative
to — while fetching blobs on demand.
- `fetch_and_reset` deepens a pre-existing shallow checkout once, up front,
rather than letting the push fail later with work on the line.
Fetch stderr is now redacted too; it can echo the remote URL.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
d90a42b759 |
fix: three gaps the P0 validation runs exposed
Validating P0 against production found one bug in each of the three pieces, none of which any test would have caught. **The scanners were installed but not allow-listed.** Mission 019fc058's condition asked for a gitleaks result; `gitleaks detect` came back `ran=false`, and the judge said it could not verify. P0.3 put the binaries in the image and never added them to `evaluator_tools::ALLOWED_PROGRAMS`, so the judge could not invoke the tools installed for it. Adds gitleaks, trivy, semgrep and `which`. **Every `continue` after a fire claim leaked the claim.** Introduced by the scheduler fix itself: the orphan-agent and empty-action paths skipped `complete_fire`, so the row stayed `claimed` — which reads as a crash mid-fire, meaning the routine is re-claimed forever and the table grows one stuck row per occurrence. Observed in production: five `claimed` rows, no dispatch, no `routine_runs`. Both paths now settle with a reason, and log it. **The agent writes its own identity files into the user's repository.** `workspace.path` is pinned to the repo root, so the runtime drops AGENTS.md, HEARTBEAT.md, IDENTITY.md, MEMORY.md, SOUL.md, TOOLS.md and USER.md into the checkout — SOUL.md opens "Who You Are / You're not a chatbot." Two consequences: every mission's tree is permanently dirty, so a `done_when` about a clean tree can never pass; and P1's `git add -A` would have committed the agent's SOUL.md into someone's repository and pushed it. The P1 deny-list covered build artifacts and would not have caught this. Fixed by writing the names to `.git/info/exclude` after clone — local to the checkout, never itself a change, and it suppresses only *untracked* files, so a repo that genuinely tracks its own AGENTS.md still reports modifications to it. Idempotent, and preserves any pre-existing exclude. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar Sobh
|
1e91a19707 |
missions: fix repo checkout for retries + tokenize git.redclaw.dev clones
- mission_orchestrator: run ensure_checkout BEFORE the team_id short-circuit. Previously, a re-launched or retried mission bailed out at the team_id=already-bound guard and skipped repo checkout entirely, so agents ran against an empty workspace. - mission_workspace: inject GITEA_TOKEN into git.redclaw.dev URLs so clone auth works from the server container. Redact any token echoed back on failure. - refresh buttons on MissionCanvas + MissionsList now spin the icon while loading so clicks are visibly acknowledged. - refresh-spinner keyframe added to motion.css. Requires operator on gw-04: sudo chown 65532:65532 /var/lib/clawmates-missions (applied 2026-07-21 pre-commit). |
||
|
Omar Sobh
|
d8c8793c4a |
ci fixes: cargo fmt, eslint entities, max-lines split
CI on
|
||
|
Omar Sobh
|
214d0c5e9f |
task #25: per-mission repo checkout on mission launch
Closes the follow-up gap flagged when task #23 landed. security_scan and benchmark_runner now exec against $CLAWMATES_MISSIONS_ROOT/ {mission_id}/repo — this commit is what actually puts a repo there. - crates/cm-api/src/mission_workspace.rs — new module. ensure_checkout(pool, workspace_id, mission_id): * mission with no repo_id → Ok(None), no-op * repo cloned into $ROOT/{id}/repo (--depth 1) * dir already a git repo → fetch + reset --hard origin/{branch} (idempotent — every launch brings the tree in sync with the remote default_branch) Auth uses the process's ambient git credential setup (SSH agent / .netrc / helper). Tokens deliberately not embedded in URLs. - crates/cm-api/src/mission_orchestrator.rs — on_launch calls ensure_checkout after team materialization + team_id bind. Non-fatal: clone failures log and continue so research_only missions (no repo needed) don't get blocked. - deploy/compose/docker-compose.yml — new named volume missions_workspaces mounted at /var/lib/clawmates-missions on both the server (writer) and where the clawmates-runtime container will mount it (reader for docker exec). CLAWMATES_ MISSIONS_ROOT + CLAWMATES_RUNTIME_CONTAINER env vars set on the server so mission_workspace + exec_target read the same canonical values. The scan/bench trigger buttons now actually produce findings once you (a) run a mission whose repo_id is set, (b) have the clawmates-runtime container bind-mounting missions_workspaces at /var/lib/clawmates-missions. Verified: SQLX_OFFLINE=true cargo check -p cm-api + cargo test -p cm-api --test mission_orchestrator both green. |