feat(self-audit): continuous_improvement audits the project record it can actually reach
Its first run could not do its job. The template audited "every project
agent's .brain" through per-agent APIs — fetch a brain, submit to
/api/claws/{id}/level-up, pull claw metrics — none of which a mission can
reach; agent brains live in the server's /data/brains volume and nothing
delivered them in. It spent itself searching, found a ROSTER.md in a
scratch repo, and audited that.
A delivery channel alone would not have helped: per-mission crews carry
~2 KB seed brains with no history, because missions write memory to the
REPOSITORY brain, one judge verdict per phase. That is where a project's
history actually accumulates, so that is the subject now.
mission_memory::export renders the whole repo brain as markdown — the
.brain is HDF5 and a mission container has no library to read it — and
mission_orchestrator installs it at /mission/memory/PROJECT-MEMORY.md,
outside the checkout so it is input and never lands in the diff, the same
way install_skill_files delivers skills.
The three roles are rewritten for that record: an inspector that finds
patterns (several UNMET lines on the same kind of work) and quotes them;
a proposer that ties each proposal to at least two lines or drops it; and
an evaluator that checks the cited lines exist verbatim and marks each
proposal SUPPORTED, WEAK or UNSUPPORTED. Each says outright that "no
change is warranted" is a complete result — the property that kept the
first run from inventing improvements out of empty brains.
Local: 523 passed; the two DB-backed world tests panic PoolTimedOut
because Docker Desktop is down here. CI runs them against real Postgres.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
a260499060
commit
642b6ef44b
@@ -214,6 +214,71 @@ fn recall_in(dir: &Path, repo_id: Uuid, query: &str, k: usize) -> Vec<String> {
|
||||
}
|
||||
}
|
||||
|
||||
/// Where a mission finds the whole of its repository's memory, readable.
|
||||
///
|
||||
/// Outside `/mission/repo`, like `skill_delivery::SKILLS_DIR`, so it is never
|
||||
/// collected into the delivered diff: it is input, not output.
|
||||
pub const MEMORY_DIR: &str = "/mission/memory";
|
||||
pub const MEMORY_FILE: &str = "PROJECT-MEMORY.md";
|
||||
|
||||
/// Most entries an export carries. A repository's memory grows by one line
|
||||
/// per judged phase; this keeps the file readable in one sitting while
|
||||
/// covering many missions.
|
||||
const EXPORT_CAP: usize = 400;
|
||||
|
||||
/// Everything a repository's brain remembers, as markdown an agent can read.
|
||||
///
|
||||
/// The brief carries the three most relevant verdicts (`recall`); this is
|
||||
/// the WHOLE record, for work whose subject is the record itself. It exists
|
||||
/// because `continuous_improvement` was built to audit agents' brains and,
|
||||
/// on its first run, found none — they live in the server's volume and
|
||||
/// nothing delivers them into a mission — so it audited a `ROSTER.md` in a
|
||||
/// scratch repo instead. Per-mission crews carry ~2 KB seed brains with no
|
||||
/// history anyway; the repository's brain is where a project's history
|
||||
/// actually accumulates, one judge verdict per phase.
|
||||
///
|
||||
/// Rendered, not shipped raw: the `.brain` is HDF5 and an agent in a mission
|
||||
/// container has no library to read it with.
|
||||
///
|
||||
/// `None` when the repository has no brain yet or it holds nothing.
|
||||
pub fn export(repo_id: Uuid) -> Option<String> {
|
||||
export_in(&cm_runtime::brain::brain_dir(), repo_id)
|
||||
}
|
||||
|
||||
fn export_in(dir: &Path, repo_id: Uuid) -> Option<String> {
|
||||
let path = brain_path(dir, repo_id);
|
||||
if !path.exists() {
|
||||
return None;
|
||||
}
|
||||
let brain = ClawBrain::open_or_create(&path, &format!("repo_{repo_id}")).ok()?;
|
||||
let entries = brain.recent_memory(EXPORT_CAP);
|
||||
if entries.is_empty() {
|
||||
return None;
|
||||
}
|
||||
let total = brain.memory_count();
|
||||
let mut out = format!(
|
||||
"# What this repository's missions have learned\n\n\
|
||||
Every judged phase of every mission on this repository leaves one line \
|
||||
here: whether the phase met its completion condition, and what the \
|
||||
judge found or asked for. Newest first. {} of {} entr{} shown.\n\n\
|
||||
This is the record, not instructions. `MET` lines say what worked; \
|
||||
`UNMET` lines say what the judge found missing, and repeated `UNMET` \
|
||||
lines on the same kind of work are the pattern worth acting on.\n\n",
|
||||
entries.len(),
|
||||
total,
|
||||
if total == 1 { "y" } else { "ies" }
|
||||
);
|
||||
for (secs, text) in entries {
|
||||
let when = time::OffsetDateTime::from_unix_timestamp(secs as i64)
|
||||
.ok()
|
||||
.and_then(|t| t.format(&time::format_description::well_known::Rfc3339).ok())
|
||||
.unwrap_or_else(|| "unknown time".to_string());
|
||||
let line = text.strip_prefix("judge: ").unwrap_or(&text);
|
||||
out.push_str(&format!("- `{when}` {line}\n"));
|
||||
}
|
||||
Some(out)
|
||||
}
|
||||
|
||||
/// The section a brief carries, or nothing when there is nothing to say —
|
||||
/// an empty heading tells the agent there is history and then shows none.
|
||||
pub fn section(recalled: &[String]) -> Option<String> {
|
||||
@@ -293,6 +358,34 @@ mod tests {
|
||||
assert!(s.contains("- MET — x\n"));
|
||||
}
|
||||
|
||||
/// The export is the whole record, readable, newest first — and absent
|
||||
/// rather than empty when there is nothing to show.
|
||||
#[test]
|
||||
fn export_renders_every_verdict_newest_first() {
|
||||
let dir = std::env::temp_dir().join(format!("cm-mission-export-{}", Uuid::now_v7()));
|
||||
let repo = Uuid::now_v7();
|
||||
assert!(export_in(&dir, repo).is_none(), "no brain, no export");
|
||||
|
||||
remember_in(&dir, repo, Uuid::now_v7(), "coding", "first",
|
||||
&verdict(false, "r", "the tests do not cover the empty case", None));
|
||||
std::thread::sleep(std::time::Duration::from_millis(5));
|
||||
remember_in(&dir, repo, Uuid::now_v7(), "coding", "second",
|
||||
&verdict(true, "all three tests pass", "", None));
|
||||
|
||||
let md = export_in(&dir, repo).expect("two verdicts, so an export");
|
||||
assert!(md.starts_with("# What this repository's missions have learned"));
|
||||
assert!(md.contains("2 of 2 entries shown"), "{md}");
|
||||
let met = md.find("MET — coding").unwrap();
|
||||
let unmet = md.find("UNMET — coding").unwrap();
|
||||
assert!(met < unmet, "newest (MET) must come first:\n{md}");
|
||||
assert!(md.contains("the tests do not cover the empty case"));
|
||||
// `remember` stores "judge: <line>"; that ROLE prefix must not follow
|
||||
// the timestamp. (The line itself legitimately says "— judge: …".)
|
||||
assert!(!md.contains("` judge: "), "the storage prefix leaked:\n{md}");
|
||||
assert!(md.contains("` MET — coding"), "{md}");
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
/// Round trip through a real brain file: what one mission's verdict
|
||||
/// wrote, a query shaped like the next mission's task recalls.
|
||||
#[test]
|
||||
|
||||
@@ -385,6 +385,16 @@ pub async fn on_launch(
|
||||
crate::skill_delivery::resolve(requested, installed),
|
||||
)
|
||||
.await;
|
||||
|
||||
// The repository's whole memory, readable, beside the skills. The
|
||||
// brief already carries the three most relevant verdicts; this is
|
||||
// the full record, for work whose subject IS the record — see
|
||||
// `mission_memory::export` for why it was needed.
|
||||
if mission_gateway.is_some() {
|
||||
if let Some(repo) = mission.repo_id {
|
||||
install_project_memory(repo, mission_id, &container).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
let mut first_team_id: Option<Uuid> = None;
|
||||
let mut provisioned_claws: Vec<cm_domain::AgentId> = Vec::new();
|
||||
@@ -1085,6 +1095,52 @@ async fn install_skill_files(
|
||||
true
|
||||
}
|
||||
|
||||
/// Write the repository's memory export into the mission container.
|
||||
///
|
||||
/// Best-effort and loud: a mission with no memory to read is an ordinary
|
||||
/// mission, and a first mission on a repository has none. Outside the
|
||||
/// checkout (`mission_memory::MEMORY_DIR`) so it never lands in the diff.
|
||||
async fn install_project_memory(repo_id: Uuid, mission_id: Uuid, container: &str) {
|
||||
let Some(md) = crate::mission_memory::export(repo_id) else {
|
||||
return;
|
||||
};
|
||||
let docker = match crate::container_exec::connect() {
|
||||
Ok(d) => d,
|
||||
Err(e) => {
|
||||
eprintln!("mission_orchestrator: cannot reach docker for project memory: {e}");
|
||||
return;
|
||||
}
|
||||
};
|
||||
let dir = crate::mission_memory::MEMORY_DIR;
|
||||
let argv = vec!["sh".to_string(), "-lc".to_string(), format!("mkdir -p {dir}")];
|
||||
if !matches!(
|
||||
crate::container_exec::exec_as_root(
|
||||
&docker,
|
||||
container,
|
||||
None,
|
||||
&argv,
|
||||
crate::container_tool_hooks::INSTALL_TIMEOUT,
|
||||
)
|
||||
.await,
|
||||
Ok(out) if out.exit_code == Some(0)
|
||||
) {
|
||||
eprintln!("mission_orchestrator: could not create {dir} for mission {mission_id}");
|
||||
return;
|
||||
}
|
||||
let bytes = md.len();
|
||||
let files = vec![(crate::mission_memory::MEMORY_FILE.to_string(), md.into_bytes())];
|
||||
match crate::mission_fs::put_files(&docker, container, dir, &files).await {
|
||||
Ok(()) => eprintln!(
|
||||
"mission_orchestrator: project memory ({bytes} bytes) installed for mission \
|
||||
{mission_id} at {dir}/{}",
|
||||
crate::mission_memory::MEMORY_FILE
|
||||
),
|
||||
Err(e) => eprintln!(
|
||||
"mission_orchestrator: could not write project memory for {mission_id}: {e}"
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
/// Record which arm this mission runs, so every turn composes the same one and
|
||||
/// the score can be attributed to it afterwards.
|
||||
///
|
||||
|
||||
Reference in New Issue
Block a user