feat(fleet): B1 — microvm runtime kind and KVM placement predicate

Phase B step 1, on top of the B0 spike that proved microVMs boot here.

KVM is a HARD predicate, not a preference. gw-04 — where every mission
runs today — is itself a VM without nested virtualisation and has no
/dev/kvm, so a microvm mission landing there cannot start at all. The
scheduler therefore has to be able to tell nodes apart, which means the
node has to report what it can host.

Nodes gain a `capabilities` jsonb, populated from a probe on the node
rather than from configuration: /dev/kvm either exists there or it does
not, and nothing on the server can make it appear. The probe OPENS the
device rather than stat-ing it, because it can exist while being
unopenable (wrong group, or a container without the device passed
through) — which is precisely how firecracker will fail.

`microvm` requires BOTH kvm and a firecracker binary. A node with KVM
but no binary looks capable by the obvious test and fails at launch; a
node with the binary but no KVM is gw-04.

Placement fails the launch when no capable node exists, rather than
letting a mission sit in 'running' with nowhere to run. An explicit
target_node_id is treated as a request, not a guarantee — it is honoured
only if that node actually reports the capability.

`capabilities` defaults to '{}' NOT NULL so a node that has never
reported fails every predicate: an unqueried node and an incapable node
must be indistinguishable to the scheduler, because scheduling onto a
node whose abilities are unknown is how you get a mission that cannot
start and does not say why. The report replaces rather than merges, so a
capability the node has LOST disappears instead of leaving a stale true.

Co-Authored-By: Claude Opus 5 <[email protected]>

crates/bins/clawmates-node/src/main.rs

@@ -131,6 +131,14 @@ async fn run(ws_url: &str) -> Result<(), Box<dyn std::error::Error>> {
         if tools_tx.send(frame).is_err() {
             break;
         }
+        // What this node can HOST, as opposed to what it has installed. The
+        // scheduler needs it to place microVM missions, and the node is the
+        // only honest source: /dev/kvm either exists here or it does not, and
+        // no amount of configuration on the server can make it appear.
+        let caps = json!({ "t": "node_capabilities", "capabilities": probe_capabilities() });
+        if tools_tx.send(caps.to_string()).is_err() {
+            break;
+        }
         std::thread::sleep(Duration::from_secs(900));
     });
 
@@ -241,6 +249,59 @@ fn heartbeat(sys: &mut System) -> String {
 /// Probe installed dev-tool versions: for each tool, find its binary across the
 /// usual bin dirs and read `--version`. Returns `{ tool: "x.y.z", … }` for the
 /// ones found. Probes `kimi-cli` (the real uv tool), not the `kimi` API wrapper.
+/// What this node can HOST — the inputs to placement predicates.
+///
+/// Distinct from [`probe_tools`], which reports what is *installed* for the
+/// operator to see and update. This answers "may the scheduler put a microVM
+/// mission here", and the answer is a property of the hardware: gw-04 is
+/// itself a VM without nested virtualisation and has no `/dev/kvm`, so it can
+/// never host one however it is configured.
+///
+/// Every value is probed, never assumed. A capability that is merely expected
+/// is the same as a capability that is absent, right up until a mission is
+/// scheduled onto a node that cannot run it.
+fn probe_capabilities() -> Value {
+    // The device node is necessary but not sufficient — it can exist while
+    // being unopenable (wrong group, or a container without the device
+    // passed through). Try to open it, because that is what firecracker does.
+    let kvm = std::fs::OpenOptions::new()
+        .read(true)
+        .write(true)
+        .open("/dev/kvm")
+        .is_ok();
+
+    let firecracker = std::process::Command::new("firecracker")
+        .arg("--version")
+        .output()
+        .ok()
+        .filter(|o| o.status.success())
+        .and_then(|o| {
+            String::from_utf8_lossy(&o.stdout)
+                .lines()
+                .next()
+                .map(|l| l.trim().to_string())
+        });
+
+    capabilities_from(kvm, firecracker.as_deref())
+}
+
+/// Shape the capability report from probe results.
+///
+/// Split from [`probe_capabilities`] so the rule can be tested without a
+/// `/dev/kvm` to open — the machine running the tests is usually the one that
+/// cannot host a microVM.
+fn capabilities_from(kvm: bool, firecracker: Option<&str>) -> Value {
+    json!({
+        "kvm": kvm,
+        "firecracker": firecracker,
+        // BOTH must hold. A node with KVM but no firecracker binary looks
+        // capable by the obvious test and fails at launch; a node with the
+        // binary but no KVM is gw-04. Computed here rather than in the
+        // scheduler so the rule sits next to the probe that feeds it.
+        "microvm": kvm && firecracker.is_some(),
+    })
+}
+
 fn probe_tools() -> Value {
     let home = std::env::var("HOME").unwrap_or_default();
     let dirs = [
@@ -1274,3 +1335,40 @@ fn ensure_tmux() {
         eprintln!("tmux not found (auto-install unavailable) — host terminal will use a plain shell; `apt install tmux` for resumable sessions");
     }
 }
+
+#[cfg(test)]
+mod capability_tests {
+    use super::*;
+
+    #[test]
+    fn microvm_needs_both_kvm_and_firecracker() {
+        assert_eq!(
+            capabilities_from(true, Some("Firecracker v1.16.1"))["microvm"],
+            json!(true)
+        );
+        assert_eq!(
+            capabilities_from(true, None)["microvm"],
+            json!(false),
+            "KVM without firecracker cannot host a microVM"
+        );
+        assert_eq!(
+            capabilities_from(false, Some("Firecracker v1.16.1"))["microvm"],
+            json!(false),
+            "firecracker without KVM is gw-04 — it can never host one"
+        );
+        assert_eq!(capabilities_from(false, None)["microvm"], json!(false));
+    }
+
+    /// The report replaces rather than merges server-side, so a node that has
+    /// LOST a capability must say so rather than omitting the key — an absent
+    /// key and a false one must not be distinguishable to the predicate.
+    #[test]
+    fn a_lost_capability_is_reported_false_not_omitted() {
+        let caps = capabilities_from(false, None);
+        assert!(caps.get("kvm").is_some(), "kvm must always be present");
+        assert!(
+            caps.get("microvm").is_some(),
+            "microvm must always be present"
+        );
+    }
+}