Fleet P2b: run agent sandboxes on connected nodes (RemoteDriver + placement)
ci / gates (push) Failing after 5s
ci / rust (push) Has been skipped
ci / sandbox-k8s (push) Has been skipped
ci / frontend (push) Has been skipped
ci / e2e (push) Has been skipped

Agents can now provision their sandbox on a connected fleet node instead of the
gateway host. Local stays the strict default, so existing agents are byte-for-
byte unaffected until explicitly placed elsewhere.

Security parity: the daemon links the REAL cm-sandbox DockerDriver and runs the
typed container ops (sb_provision/sb_exec/sb_destroy/sb_health/sb_list) through
it — identical hardening (cap-drop ALL, seccomp, no-net, read-only, non-root) to
local sandboxes. cm-sandbox spec types are now Serialize/Deserialize so the spec
crosses the channel.

- cm-api: RemoteDriver (impl SandboxDriver over the node channel) + HubDriverProvider
  (impl cm_runtime::NodeDriverProvider, hands out a driver only for connected
  nodes via a sync online set) + NodeHub.call/is_connected. AppState.with_node_hub
  so the hub is shared with the placement provider.
- cm-runtime SandboxManager: driver_for(node_id) routes by the recorded
  agent_containers.node_id (local default = existing driver, identical path);
  placement_node() reads the workspace setting and falls back to local if the
  node is offline; exec/release route accordingly. NodeDriverProvider trait.
- DB: 0020_workspace_placement + repo (for_agent/get/set/clear).
- main.rs: build the NodeHub first; inject HubDriverProvider into the agent
  manager + share the hub with AppState.
- API+UI: GET/PUT /api/fleet/placement + a "Run agents on: Local / <node>"
  selector in the Fleet overview.

Note: a node must be able to pull the agent image (the daemon docker-pulls it);
interactive PTY for agent containers on remote nodes is not wired (Terminal app
stays local) — the in-dashboard node shell already covers host access.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
Omar Sobh
2026-06-24 12:48:50 -07:00
co-authored by Claude Opus 4.8
parent 33aa9c0693
commit fb59378aa2
30 changed files with 5740 additions and 54 deletions
+37 -1
View File
@@ -4,12 +4,48 @@
use axum::extract::State;
use axum::Json;
use cm_db::repo::fleet_tailscale;
use cm_db::repo::{fleet_tailscale, nodes, workspace_placement};
use cm_domain::NodeId;
use serde::Deserialize;
use serde_json::{json, Value};
use uuid::Uuid;
use crate::{ApiError, AppState, Authed};
/// `GET /api/fleet/placement` — the workspace's default placement node (or null).
pub async fn placement_get(
State(state): State<AppState>,
Authed(user): Authed,
) -> Result<Json<Value>, ApiError> {
let node = workspace_placement::get(&state.pool, user.workspace_id).await?;
Ok(Json(json!({ "node": node })))
}
#[derive(Deserialize)]
pub struct PlacementReq {
pub node: String,
}
/// `PUT /api/fleet/placement` — set where new agent sandboxes provision. "local"
/// (or empty) reverts to the gateway host; a node id must belong to the caller.
pub async fn placement_set(
State(state): State<AppState>,
Authed(user): Authed,
Json(req): Json<PlacementReq>,
) -> Result<Json<Value>, ApiError> {
let node = req.node.trim();
if node.is_empty() || node == "local" {
workspace_placement::clear(&state.pool, user.workspace_id).await?;
return Ok(Json(json!({ "ok": true, "node": "local" })));
}
let nid = NodeId::from(node.parse::<Uuid>().map_err(|_| ApiError::NotFound)?);
nodes::get(&state.pool, nid, user.workspace_id)
.await?
.ok_or(ApiError::NotFound)?;
workspace_placement::set(&state.pool, user.workspace_id, node).await?;
Ok(Json(json!({ "ok": true, "node": node })))
}
#[derive(Deserialize)]
pub struct ConnectReq {
#[serde(rename = "apiKey")]