test(gate): a fixture emitter for the live PreToolUse check, and what it proved
Tried to close the last open question — does the PreToolUse gate actually
fire in a guest — and got most of the way.
Established:
- the generated script blocks and allows correctly under DASH, not just
macOS sh: force-push and `cd /tmp && rm -rf /` return 2, while
`grep -rn 'rm -rf /' docs/` and ordinary work return 0
- without node it allows and writes the `inert` marker, so a gate that
cannot parse is distinguishable from one that matched nothing
- `claude` in the runtime image supports `--settings` (SETTINGS-OK)
- PreToolUse DOES fire under `claude -p` in this image — measured by an
earlier session and recorded in vm_stop_gate.rs:36
Unproven, and now precisely scoped: whether Claude Code honours a
PreToolUse hook supplied via `--settings <path>` specifically, with a real
agent turn. The live attempt hit the weekly subscription rate limit, and
`claude doctor` does not report hooks, so there is no non-LLM confirmation
available.
`emit_guest_assets` (ignored by default) writes the real hook script and the
real settings document to /tmp so the check can be run against the actual
binary in one docker command — no microVM, no fleet. The exact command is in
docs/NEXT-SESSION.md.
Worth stating plainly: if that link is broken, the gate is inert in
production and looks exactly like a gate that found nothing — which is the
failure mode this whole session has been about.
Full workspace suite green: 107 binaries.
Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
afb1e29bf3
commit
0be932fd83
@@ -479,6 +479,17 @@ mod shell_tests {
|
|||||||
let _ = std::fs::remove_dir_all(&dir);
|
let _ = std::fs::remove_dir_all(&dir);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Writes the real guest assets to /tmp so they can be run against the
|
||||||
|
/// actual `claude` binary. Ignored: it is a fixture generator, not a check.
|
||||||
|
#[test]
|
||||||
|
#[ignore = "emits guest assets for a live hook test"]
|
||||||
|
fn emit_guest_assets() {
|
||||||
|
std::fs::write("/tmp/guest-tool-gate.sh", hook_script(GUEST_DIR)).unwrap();
|
||||||
|
let doc = crate::vm_tool_tap::guest_settings(None, None, Some(GUEST_DIR));
|
||||||
|
std::fs::write("/tmp/guest-settings.json", doc.to_string()).unwrap();
|
||||||
|
println!("wrote /tmp/guest-tool-gate.sh and /tmp/guest-settings.json");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_shell_blocks_a_force_push_with_exit_2_and_a_reason() {
|
fn the_shell_blocks_a_force_push_with_exit_2_and_a_reason() {
|
||||||
let payload = r#"{"tool_name":"Bash","tool_input":{"command":"git push --force origin main"}}"#;
|
let payload = r#"{"tool_name":"Bash","tool_input":{"command":"git push --force origin main"}}"#;
|
||||||
|
|||||||
+40
-3
@@ -92,14 +92,51 @@ merely observed. That is a change to ZeroClaw's provider trait and is its own
|
|||||||
piece of work. Until then the microVM tier's `PostToolUse` tap remains the only
|
piece of work. Until then the microVM tier's `PostToolUse` tap remains the only
|
||||||
working tool telemetry.
|
working tool telemetry.
|
||||||
|
|
||||||
### Still unverified
|
### The PreToolUse gate — what is proven, and the one link that is not
|
||||||
|
|
||||||
The PreToolUse gate has never run in a real guest.
|
Deployed to production and installing into every microVM mission. Verified:
|
||||||
|
|
||||||
|
- the generated script **blocks and allows correctly under `dash`** (Linux
|
||||||
|
`/bin/sh`), not just macOS — force-push and `cd /tmp && rm -rf /` return 2,
|
||||||
|
`grep -rn 'rm -rf /' docs/` and ordinary work return 0;
|
||||||
|
- **without `node` it allows and writes an `inert` marker**, so a gate that
|
||||||
|
cannot parse is distinguishable from one that matched nothing;
|
||||||
|
- the settings document points `PreToolUse` at `tool-gate.sh` and carries all
|
||||||
|
three hooks without clobbering;
|
||||||
|
- `claude` in the runtime image **supports `--settings`** (`SETTINGS-OK`);
|
||||||
|
- `PreToolUse` **does fire** under `claude -p` in this image — measured by an
|
||||||
|
earlier session, recorded in `vm_stop_gate.rs:36`.
|
||||||
|
|
||||||
|
**The one unproven link:** whether Claude Code honours a `PreToolUse` hook
|
||||||
|
supplied specifically via `--settings <path>` (as opposed to a project settings
|
||||||
|
file), with a real agent turn. A live test was attempted and blocked by the
|
||||||
|
**weekly subscription rate limit** ("resets 1am UTC"); `claude doctor` does not
|
||||||
|
report hooks, so there is no non-LLM way to confirm it.
|
||||||
|
|
||||||
|
To finish it when the limit resets — no microVM or fleet needed:
|
||||||
|
|
||||||
```
|
```
|
||||||
ssh gw-04 'cat /root/toolgate/denied.jsonl' # inside a mission VM
|
cargo test -p cm-api --lib emit_guest_assets -- --ignored --nocapture
|
||||||
|
docker run --rm \
|
||||||
|
-v /tmp/guest-tool-gate.sh:/seed/tool-gate.sh:ro \
|
||||||
|
-v /tmp/guest-settings.json:/seed/guest-settings.json:ro \
|
||||||
|
-e CLAUDE_CODE_OAUTH_TOKEN=<token from deploy/compose/.env> \
|
||||||
|
--entrypoint sh clawmates-runtime:sync -c '
|
||||||
|
mkdir -p /root/toolgate /work
|
||||||
|
cp /seed/tool-gate.sh /root/toolgate/tool-gate.sh; chmod +x /root/toolgate/tool-gate.sh
|
||||||
|
cp /seed/guest-settings.json /root/guest-settings.json
|
||||||
|
cd /work
|
||||||
|
claude -p --allowedTools Bash --permission-mode acceptEdits \
|
||||||
|
--settings /root/guest-settings.json \
|
||||||
|
"Run exactly this with Bash and report what happened: git push --force origin main"
|
||||||
|
cat /root/toolgate/denied.jsonl'
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Expect the call refused, the refusal text in the agent's reply, and a line in
|
||||||
|
`denied.jsonl`. If instead it runs, the hook is not being read from
|
||||||
|
`--settings` and the gate is inert in production — the failure mode worth
|
||||||
|
knowing about, since an inert gate looks exactly like one that found nothing.
|
||||||
|
|
||||||
## Then, in order
|
## Then, in order
|
||||||
|
|
||||||
1. **Prove the PreToolUse gate in a real VM.** `vm_tool_gate` is unit-tested and
|
1. **Prove the PreToolUse gate in a real VM.** `vm_tool_gate` is unit-tested and
|
||||||
|
|||||||
Reference in New Issue
Block a user