clawverse/clawstor
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
154 Commits 10 Branches 0 Tags
bf439319d554827c0c3d53f2e9a82dd4f5cf88e3
Commit Graph
10 Commits
Author SHA1 Message Date
Omar Sobh 2d09b4687c Phase 5b: KV refs + claw-cargo CLI (the killer feature, live) 
Ships the actual user-facing cargo build cache. Combined with Phase 5a
(fingerprint + capture + restore) + the whole Phase 2 blob substrate,
`claw-cargo build` now runs `cargo build` with a peer-cache lookup:
hit → download+restore, miss → build+capture+upload.

## What ships

### cluster/refs.rs (243 lines)

A dumb 32-byte-key → 32-byte-value directory-backed store. Used to map
fingerprints → BlobIds. Layout mirrors BlobStore:

  <root>/
    refs/<kk>/<key_hex>.ref     — 32 raw bytes
    .tmp/                        — atomic-rename staging

Public API: RefStore::open / get / put / delete / contains. All writes
atomic via tempfile + rename. Deliberately no versioning or CRDT
semantics — that's Phase 3. Every real cargo-cache lookup is a
single-key-single-value shape.

### New RPC methods

- GetRef (0x0d): payload = 32-byte RefKey; reply = 32 bytes / NotFound
- PutRef (0x0e): payload = 32-byte RefKey || 32-byte RefValue;
  reply = STREAM_STATUS_OK / error

### RpcRouter + services

- RpcRouter grows optional Arc<RefStore> via `with_ref_store`
- ClusterServices opens a RefStore alongside the BlobStore when
  `blob_store_root` is configured (co-located at `<blob_root>/refs-db`)
- `blob_store_enabled()` / `ref_store_enabled()` introspection

### claw-cargo binary (319 lines)

New bin target `claw-cargo` — thin CLI wrapping the whole stack:

  claw-cargo fingerprint --profile release --features "a,b"
    → prints the workspace fingerprint (no network)

  claw-cargo build \
    --peer <name> --peer-addr <ip:port> --tls-dir <dir> \
    --profile release --features "a,b" \
    -- --workspace=x --frozen ...
    → 1. compute fingerprint
      2. QUIC + mTLS connect to peer
      3. GetRef(fingerprint) → BlobId?
         HIT: BlobStat → BlobGetStream → restore_target → cargo build
         MISS: cargo build → capture_target → BlobPutStream → PutRef
      4. Print summary: fingerprint, hit/miss, bytes, cargo elapsed

## Live smoke test

Ran claw-cargo fingerprint on this workspace with three profile/feature
combos — got three distinct 32-byte fingerprints. Same profile+features
on the same workspace state → same fingerprint (Phase 5a's guarantee
carried through the CLI).

## Tests (14 new, all real — no mocks)

Refs store (7):
- open creates layout
- get returns None for missing
- put + get round-trips
- put overwrites prior value
- delete removes ref + reports (false on second delete)
- distinct keys produce distinct on-disk files (bucket fan-out proof)
- rejects_wrong_length_on_disk (corruption detection)

RPC (7):
- phase_5b_method_byte_encoding
- get_ref_returns_not_found_for_missing
- put_ref_stores_and_get_ref_reads_back
- put_ref_rejects_wrong_length_payload
- get_ref_rejects_wrong_length_payload
- ref_rpcs_return_not_configured_without_store
- end_to_end_put_ref_get_ref_over_real_quic — full 2-node QUIC + mTLS
  round trip proving PutRef/GetRef work at the wire level

188 tests pass. Pre-existing macOS-only failure unchanged.

File sizes (all under 1300-line ceiling):
- cluster/refs.rs: 243
- cluster/rpc.rs: 1169
- cluster/rpc/tests.rs: 1073
- cluster/services.rs: 565
- claw_cargo.rs: 319

## Where this leaves us

The distributed FS + cargo cache is functionally complete for the
happy path:

  Node A builds clawverse for the first time
  → cargo build (50 min cold)
  → capture_target (a few seconds)
  → push to node B via BlobPutStream (network-bound)
  → PutRef(fingerprint → BlobId)

  Node B on the same workspace state runs `claw-cargo build …`
  → compute_fingerprint (ms)
  → GetRef → hit
  → BlobGetStream (network-bound)
  → restore_target (a few seconds)
  → cargo build → sees valid deps/.fingerprint, builds only
    workspace crates (~3 min instead of 50)

Same workspace state on a third machine? Same fingerprint → same
cache hit. That's the whole design.

## Follow-on

- Phase 5c: pre-fetch on Gitea webhook so CI runners never wait
- Phase 5d: metric ticker publishes cache hit rate into gossip so
  the placement engine can bias runner scheduling toward warm nodes
- Phase 3: CRDT metadata for human-readable pins on top of raw
  32-byte refs (`clawverse:main:latest-cache` → fingerprint hex)
- Phase 6+: FUSE mount for the warm-tier git worktrees
 2026-07-11 23:36:37 -07:00
Omar Sobh 242ef527b4 Phase 5a: fingerprint + capture + restore for build-artifact cache 
The substrate for the killer feature. Given a cargo workspace, compute
a deterministic 32-byte BLAKE3 fingerprint over the inputs that
determine what artifacts should be produced, then bundle the portable
subset of `target/<profile>/` into a zstd-compressed tarball ready to
hand to BlobStore.

## Module: cluster/build_cache.rs (661 lines)

Types:
- FingerprintInputs { cargo_lock, rustc_version_verbose, cargo_config,
  rust_toolchain, profile, features, rustflags, target_triple }
- Fingerprint(32 bytes) — parallel shape to BlobId

Public API:
- FingerprintInputs::collect(workspace, profile, features) — reads
  Cargo.lock, shells out to `rustc --version --verbose`, reads
  optional config files, extracts host triple, sorts+dedups features
- FingerprintInputs::compute() → Fingerprint — domain-separated
  BLAKE3 with per-field labels + null separators so field boundaries
  can't collide
- capture_target(target_dir) → zstd-tarball bytes
- restore_target(bytes, target_dir) → unpacks
- capture_workspace(workspace, profile) — resolves target dir
- compute_workspace_fingerprint(workspace, profile, features) —
  returns (inputs, fingerprint)

Captured: deps/, .fingerprint/, build/, examples/, plus small
top-level files (.cargo-lock, .rustc_info.json, CACHEDIR.TAG).
Explicitly NOT captured: incremental/ (per-machine, not portable —
tied to absolute paths + rustc state; restoring on another host
silently corrupts the build).

Determinism guarantees:
- HeaderMode::Deterministic on the tar builder — identical trees
  produce byte-identical tarballs (proven by
  `capture_yields_identical_bytes_for_identical_input`)
- follow_symlinks(false) — symlinks archived as symlinks, not their
  targets, so the fingerprint doesn't drift with symlink destinations
- Features sorted + deduped so `["b","a"]` and `["a","b"]` hash the same
- Missing optional files treated as empty strings so `absent ==
  empty` (add-then-remove doesn't churn the hash)

Deps added: tar 0.4, zstd 0.13.

## Tests (18 new, all real filesystem — no mocks)

Fingerprint (7):
- fingerprint_is_deterministic
- fingerprint_changes_when_cargo_lock_changes
- fingerprint_changes_when_profile_changes
- fingerprint_changes_when_features_change
- fingerprint_is_feature_order_independent (proves sort semantics)
- fingerprint_domain_separation_prevents_field_collision — swap
  content between two string fields; naive concat hasher would
  collide, ours doesn't
- fingerprint_hex_length_and_stability

Input collection (4):
- read_optional_returns_empty_for_missing
- read_optional_returns_content_for_existing
- collect_errors_when_cargo_lock_absent
- collect_reads_cargo_lock_and_computes — actually shells out to
  `rustc --version --verbose`, verifies triple extraction

Capture/restore (5):
- capture_target_errors_when_dir_missing
- capture_and_restore_round_trip_preserves_files — full 6-file
  tree including a captured example + an intentionally-excluded
  incremental/ dir; verifies incremental/ is absent after restore
- capture_yields_identical_bytes_for_identical_input — byte-equal
  tarballs across two identical source trees
- capture_skips_top_level_files_not_in_allowlist
- capture_workspace_resolves_profile_dir

End-to-end (1):
- **end_to_end_fingerprint_capture_blob_restore** — the full
  workspace → fingerprint → capture → BlobStore::put_bytes →
  BlobStore::get_bytes → restore_target loop. Proves the whole
  round-trip lands byte-equal for both captured file trees, and
  the BlobId is deterministic across runs. This is the primitive
  the cargo-cache CLI sits on top of.

174 tests pass. Pre-existing macOS-only failure unchanged.

File sizes (well under 1300-line ceiling):
- cluster/build_cache.rs: 661
- cluster.rs (submodule declarations): 279

## What this unlocks

Phase 5a is the substrate. The CLI wrapper (`claw-cargo build`) is
Phase 5b — thin glue that:
  1. Runs compute_workspace_fingerprint()
  2. Asks the peer BlobStat(fingerprint_as_blob_id)
  3. Hit → BlobGetStream + restore_target + cargo build (just the
     workspace's own crates, ~seconds)
  4. Miss → cargo build (full), then capture_target +
     push_blob_missing_chunks + PutManifest

Every piece of infrastructure that Phase 5b needs — content-addressed
blob store, streaming and partial-chunk RPC, mTLS transport, gossip-
driven peer discovery — is already merged. Phase 5b is CLI polish,
not new distributed-systems machinery.

## Follow-on

- Phase 5b: claw-cargo CLI wrapper (small — 200-400 lines)
- Phase 5c: pre-fetch on Gitea webhook (workflow triggers → daemon
  pre-warms the fingerprint on the target runner)
- Phase 3 remains a parallel track — human-readable namespace layer
  on top of raw fingerprints so operators can pin
  `clawverse:main:latest-cache` instead of a hex string
 2026-07-11 23:27:57 -07:00
Omar Sobh eab10005fd Phase 2: content-addressed blob store 
The storage substrate everything after Phase 1 sits on. Every blob is
identified by its BLAKE3 whole-content hash (BlobId); on disk it lives
as an ordered sequence of BLAKE3-hashed 4 MB chunks, so blobs that
share a prefix (two cargo target dirs with 95% of the same deps) share
storage at chunk granularity with no special detection logic.

New: cluster/blob.rs (802 lines).

Types:
- BlobId — 32-byte BLAKE3 output, hex-serialised (serde ↔ string)
- ChunkHash — same shape as BlobId but a distinct type so blob and
  chunk lookups can't accidentally swap
- BlobStat — { total_size, chunk_count }
- BlobManifest — { blob_id, total_size, chunks: Vec<ChunkHash> },
  public because Phase 2b RPC serves it directly so a receiver can
  request only the chunks it's missing
- GcReport — { chunks_scanned, chunks_removed, bytes_reclaimed }
- BlobStore — root-directory-based store

Public API:
- BlobStore::open(root)
- put_bytes(&[u8]) → BlobId
- get_bytes(&BlobId) → Option<Vec<u8>>  (verifies hash + size on read)
- contains(&BlobId) → bool
- stat(&BlobId) → Option<BlobStat>
- load_manifest(&BlobId) → Option<BlobManifest>
- delete_manifest(&BlobId) → bool   (chunks stay; orphan by GC)
- gc_orphan_chunks() → GcReport

On-disk layout:
  <root>/
    blobs/<bb>/<blob_hash>.manifest.json
    chunks/<cc>/<chunk_hash>
    .tmp/
Two-char bucket prefixes cap fan-out at 256 entries per level — safe
on a warm-tier ZFS dataset with tens of thousands of blobs.

Every write is atomic (tmp file + rename on same filesystem).
Every chunk write is a no-op if the file already exists — same
content across two put()s stores exactly one physical copy.

Correctness:
- Reads verify each chunk against its hash + recompute the whole-blob
  hash before returning; a bit-flipped chunk raises "chunk hash
  mismatch" instead of silently corrupting the answer.
- delete_manifest is the only deletion primitive; chunks are only
  ever removed by gc_orphan_chunks after a full manifest scan proves
  they're unreferenced.

Dep: blake3 = "1".

Tests (21 new, all real filesystem, no mocks):
- BlobId/ChunkHash hex round-trip + serde JSON
- BlobId::from_hex rejects wrong-length + non-hex input
- open creates blobs/, chunks/, .tmp/
- put + get round trip: small, empty, 10 MB (3 chunks)
- put is deterministic (same bytes → same BlobId every time)
- put is idempotent (writing twice → exactly one manifest file)
- Different content → different BlobId
- shared_chunks_are_stored_only_once: two blobs sharing a 4 MB prefix
  produce exactly 3 chunk files, not 4
- get_returns_none_when_missing / contains false / stat None
- delete_manifest keeps chunks (proven by counting chunk files)
- delete_manifest on missing returns false
- gc_reclaims_orphan_chunks_but_keeps_referenced: put 2 blobs, delete
  one manifest, GC removes exactly the orphaned chunk, keeps
  live A readable
- gc_on_empty_store_reports_zero
- corrupted_chunk_detected_on_read: rewrite a chunk with garbage →
  get_bytes errors with "chunk hash mismatch"
- load_manifest round-trips the chunk list

116 tests pass. Pre-existing macOS-only failure unchanged.

File size: cluster/blob.rs = 802 lines (ceiling 1300).

Follow-on Phase 2 cuts:
- 2b: RPC methods BlobStat / BlobGet / BlobPut, wired into RpcRouter
  and served over the QUIC transport built in Phase 1c-1e.
- 2c: streaming put/get (AsyncRead / AsyncWrite variants) for
  many-GB build artifacts.

Phase 3 (metadata + CRDTs) can start independently — this store is the
substrate the fingerprint cache in Phase 5 layers onto.
 2026-07-11 22:32:59 -07:00
Omar Sobh 916add37df Phase 1d: persistent NodeIdentity + FleetCa + fleet-ca CLI 
Closes out Phase 1. A production operator can now cut a fleet CA,
sign per-node leaves, drop the resulting PEMs at
/etc/claw-store/tls/, point [cluster.tls] at them, and the daemon
loads real mTLS material on startup — no more ephemeral in-process
CA hack.

New public API in cluster::transport:
- FleetCa::generate(cn) — new self-signed root CA
- FleetCa::save(dir) / FleetCa::load(dir) — round-trip PEM
- FleetCa::sign_leaf(name) — mint an in-memory NodeIdentity
- FleetCa::sign_leaf_to_pem(name, out_dir) — write ca.crt + node.crt +
  node.key (node.key at 0o600 on Unix)
- NodeIdentity::from_pem_files(ca, cert, key) — production load path
- NodeIdentity::from_pem_dir(dir) — canonical filename layout
- NodeIdentity::from_cluster_config(cfg) — pick up [cluster.tls] paths

Manual Debug for FleetCa redacts the private key.

Config extension:
- [cluster.tls] ca_cert / node_cert / node_key (all PathBuf).
- Optional at the top level; callers that need mTLS surface a clear
  error when it's absent.

Deps:
- rcgen features += "x509-parser" (for FleetCa::load's from_ca_cert_pem).
- rustls-pemfile 2 (parse PEM back into DER for rustls).

CLI (new commands; short-circuit config load so they run on fresh
boxes without /etc/claw-store/config.toml):
- fleet-ca-init --dir <dir> [--cn <name>]
    generates ca.crt + ca.key (both 0o600).
- fleet-ca-sign --ca-dir <dir> --node <name> --out-dir <dir>
    writes ca.crt + node.crt + node.key (node.key at 0o600).
- cluster-ping now accepts --tls-dir <dir> to load persistent
  NodeIdentity from disk (produced by fleet-ca-sign).

Tests (8 new, all real — no mocks, real filesystem, real TLS handshake):
- fleet_ca_rejects_empty_common_name
- fleet_ca_save_and_load_round_trip_preserves_signing (asserts 0o600
  on ca.key)
- fleet_ca_load_errors_when_files_missing
- sign_leaf_to_pem_writes_all_three_files_with_correct_permissions
  (asserts 0o600 on node.key)
- persistent_identity_round_trips_through_disk_and_pings — end-to-end:
  cut CA on disk, reload it, sign two leaves via sign_leaf_to_pem,
  reload them via from_pem_dir, run real QUIC ping/pong. This is the
  operator flow.
- node_identity_from_cluster_config_errors_without_tls_section
- node_identity_from_cluster_config_loads_pem_paths
- from_pem_files_errors_on_missing_ca_file

80 tests pass. Pre-existing macOS-only hot test unchanged.

Also verified live CLI smoke test:
  fleet-ca-init → ca.crt + ca.key at 0o600
  fleet-ca-sign → ca.crt + node.crt + node.key at 0o600
  Files parse as valid X.509.

File sizes (all under 1300-line ceiling):
- cluster/transport.rs: 916
- cluster/gossip.rs: 576
- cluster.rs: 275
- config.rs: 503
- main.rs: 662

Phase 1 complete. Next up:
- Phase 1e (daemon integration): gossip + QUIC RPC server wired into
  claw-store daemon; hot-tier metrics periodically pushed; a real
  PeerStatus RPC alongside ping.
- Phase 2: content-addressed blob store (BLAKE3 chunking, put/get).
 2026-07-11 22:04:54 -07:00
Omar Sobh 4d93f955c5 Phase 1c: QUIC transport with fleet-CA mTLS + cluster-ping 
Wraps quinn 0.11 in cluster/transport.rs with rustls 0.23 + rcgen 0.13
for identity management. Every peer connection is mTLS: both sides must
present certs signed by the shared fleet CA, and rustls verifies the
peer's cert SAN matches the requested server name.

Public API on `cluster::transport`:
- `NodeIdentity` (cert chain + private key + trusted CA)
- `NodeIdentity::generate_test_pair(a, b)` — ephemeral CA + two signed
  leaves; shape matches the production PEM-file loader (Phase 1d)
- `QuicServer::bind(addr, identity)` — bind mTLS-enforced listener
- `QuicServer::accept()` — accept one connection (Option<Result<_>>)
- `QuicClient::new(local_addr, identity)` — build client endpoint
- `QuicClient::connect(peer_addr, expected_name)` — outbound with SAN check
- `ping(&conn, payload)` — bidi stream RPC; server echoes as `pong:<payload>`
- `ping_handler_loop(conn)` — server-side accept/echo forever
- `CLAWSTOR_RPC_ALPN` constant, single ALPN "clawstor-rpc/1"

Config extension:
- `bind_rpc_lan`, `bind_rpc_tailscale` on `ClusterConfig` (both optional).
- Defaults: gossip port + 1 (so gossip UDP and QUIC UDP don't collide).
- Helper: `ClusterConfig::rpc_lan()`, `rpc_tailscale()`,
  `PeerEntry::rpc_lan()`, `rpc_tailscale()`.

Gossip.rs now advertises the RPC address, not the gossip address, on
the well-known `clawstor.rpc.lan` / `clawstor.rpc.tailscale` keys.

CLI:
- `cluster-ping --name <me> --peer <you> --rpc-addr <addr> [--payload X]`
  Runs a full mTLS handshake and single ping. For dev/loopback use
  today (both sides need to share a CA); persistent-identity ping
  lands in Phase 1d.

Tests (6 new, real UDP + TLS handshake, no mocks):
- generate_test_pair produces two distinct leaves that share a CA
- ping_pong_between_two_mtls_peers: real 2-node QUIC round trip with
  full mTLS chain verification, `open_bi()`/`accept_bi()`, byte-exact
  response check
- client_rejects_peer_with_wrong_ca: TLS chain verification failure
  when the server presents a cert signed by a different CA
- client_rejects_wrong_server_name: SAN mismatch is enforced
- server_binds_wildcard_and_reports_concrete_local_addr: port 0 → real
- ping_rejects_oversize_payload: MAX_MESSAGE_BYTES cap enforced client-side

Fixed-port allocator range 42000+ so transport tests don't conflict
with gossip tests (41000+).

72 tests pass. Pre-existing `hot::test_project_target_size_bytes`
macOS-only failure unchanged.

File sizes (all under 1300-line ceiling):
- cluster/transport.rs: 485
- cluster/gossip.rs: 570
- cluster.rs: 275
- config.rs: 480
- main.rs: 564

Follow-on Phase 1 cut (1d):
- Load NodeIdentity from persistent PEM files at /etc/claw-store/tls/
- Fleet CA bootstrap ceremony (rcgen → write CA cert; per-node leaf CSR)
- Daemon-level RPC server that runs alongside gossip + serves real
  operations (blob get/put, metadata sync)
- cluster-status reads from live daemon via API instead of standalone
 2026-07-11 21:50:28 -07:00
Omar Sobh 5283ab8655 Phase 1b: chitchat SWIM gossip layer + cluster-status CLI 
Wraps chitchat 0.11 in cluster/gossip.rs. Each node publishes its
zone, RPC endpoints, hot-tier usage, warm-tier project list, and
uptime under well-known kv keys; peers propagate this via gossip and
are classified alive/dead by phi-accrual failure detection.

Bootstraps from static [[cluster.peers]] seed nodes in the config.
Membership extends dynamically as nodes join or leave.

Public API on `ClusterGossip`:
- bootstrap(cluster_cfg, local_name) — start UDP gossip service
- set(key, value), set_hot_used, set_hot_max, set_warm_projects
- peers() — all known peers with typed PeerView + liveness
- peer(name) — one peer by advertised name
- peers_in_zone(zone) — live peers filtered by zone
- shutdown() — abort gossip task

CLI:
- `claw-store cluster-status --name <me> [--wait-secs N]`
  bootstraps gossip, waits for convergence, prints a peer table
  with zone, alive/dead, RPC LAN/Tailscale addresses, hot usage.

Tests (7 new):
- bootstrap_publishes_our_own_state — self-ID + advertised address
- bootstrap_fails_when_local_name_empty
- bootstrap_fails_when_no_bind_address — cluster.validate error chain
- two_node_cluster_converges_and_shares_state — REAL two-node
  in-process UDP gossip. Publish state on A, wait for phi-accrual
  liveness on B, verify all fields propagated (zone, rpc_lan, hot
  bytes, warm_projects list, fill ratio, zone filtering).
- peers_excludes_self — solo cluster reports zero peers
- peer_view_hot_fill_ratio_handles_missing_or_zero

No mocks. Real UDP transport, real chitchat runtime, real gossip
protocol. Ports allocated from a fixed range (41001+) via atomic
counter — deterministic and race-free within the test binary.

66 tests pass. Pre-existing `hot::test_project_target_size_bytes`
macOS-only failure unchanged.

File sizes (under 1300-line ceiling):
- cluster/gossip.rs: 549
- cluster.rs: 274
- main.rs: 506

Follow-on Phase 1 cuts:
- 1c: `quinn` QUIC transport + fleet-CA mTLS for RPC (separate UDP
      port so gossip and QUIC don't collide)
- 1d: daemon wire-up — gossip runs in background, hot-tier usage
      pushed periodically, `cluster-status` reads from live daemon
 2026-07-11 21:41:52 -07:00
Omar SobhandClaude Sonnet 4.6 1320d2df9e feat: v0.3.0 — HTTP handler tests + README 
- Extract build_app() from run_server() so tests can construct the
  router without binding a port (tower::ServiceExt::oneshot pattern)
- Add 10 new tests in serve::tests: validate_project_name (valid +
  invalid slugs), daemon_uptime when file absent, GET /api/status,
  GET /api/projects, POST /api/activate with invalid name, auth
  middleware (no header → 401, wrong token → 401, correct token →
  pass-through, GET bypasses auth entirely)
- Add tower + http-body-util to dev-dependencies
- Add project README covering architecture, install (Makefile),
  config options, SSH setup, snapshot schedule, CLI reference,
  HTTP API table, pinning, and troubleshooting guide
- Bump version to 0.3.0

Test suite: 39 tests, 0 failures

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
 2026-06-30 10:51:52 +00:00
Omar SobhandClaude Opus 4.7 af50adec19 feat(v0.2.0): atomic+flocked manifest, pinned projects, serve systemd unit 
The two biggest pain points coming out of the architecture review:

  1. The manifest at /var/lib/claw-store/projects.toml was the only
     piece of writable state but had no locking, no atomic writes,
     and three concurrent writers (daemon poll tick, every CLI verb,
     and the dashboard shelling out via /api/activate). Two writers
     interleaving silently dropped one of them; a crash mid-write
     left a corrupt half-written TOML that the next reader parsed
     as an empty manifest.

  2. Reboot survival: the dashboard had no systemd unit and was a
     stray hand-launched process. Architect lost its dashboard on
     todays reboot.

This commit lands:

- Manifest::update(path, FnOnce(&mut Manifest)) — locked-atomic
  load-mutate-save in one transaction. Uses libc::flock(LOCK_EX) on
  a sidecar .lock file (so the data file can be replaced by rename
  without invalidating the lock) and tempfile + persist for the
  rename. Concurrent writers serialise; readers see the previous
  state or the new state, never a torn write. Manifest::load uses
  LOCK_SH so it never races a mid-rename.

- Project.pinned: bool with #[serde(default)] so legacy manifests
  parse cleanly. hot::gc_stale_targets and hot::gc_by_space both
  skip pinned projects, with a WARN log when every remaining
  project is pinned but were still over budget — operator intent
  beats space pressure.

- claw-store pin <project> / unpin <project> CLI verbs.
  status command surfaces pin marker (📌).
  activate preserves an existing rows pinned flag so re-activating
  doesnt silently unpin.

- claw-store-serve.service systemd unit. Type=simple, Restart=
  on-failure, RestartSec=15, ProtectSystem=strict + ReadWritePaths
  =/var/lib/claw-store, ProtectHome, NoNewPrivileges, PrivateTmp.

- daemon poll tick reloads the manifest from disk at the start of
  each cycle (so CLI activations between ticks are visible) and
  routes its GC write through Manifest::update (so it cant race a
  concurrent CLI pin).

- libc + tempfile move from dev-deps into runtime deps.
- empty-manifest fallthrough on load (treat "" as default) so a
  half-written tempfile crashed pre-rename doesnt hard-fail the
  daemon next boot.

- 25 tests passing incl. new ones: legacy-toml-parses, update-
  serializes-two-sequential-writers, pinned-survives-stale-gc,
  pinned-survives-space-gc-even-when-lru.

Version bumped 0.1.0 → 0.2.0.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
 2026-06-28 12:59:41 +00:00
Omar SobhandClaude Opus 4.7 2fcfb16160 feat(serve): HTTP+SSE API + static dashboard bundle 
Adds the serve subcommand: an axum 0.7 + tower-http server on
:7700 backing the React dashboard. Routes:

  GET  /api/status   /api/projects   /api/snapshots
       /api/sync-queue   /api/hot   /api/events (SSE 5s tick)
  POST /api/activate /api/deactivate /api/sync
       /api/gc /api/snapshot

Mutating endpoints shell out to /usr/local/bin/claw-store so
all CLI logic stays single-sourced. claw-store/static/ holds
the prebuilt dashboard Vite bundle for --static-dir.

cargo_init.rs gets a minor wiring tweak so the API can read
back the managed cargo config marker.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
 2026-06-28 12:52:46 +00:00
Omar Sobh 08b7f94e4c feat: claw-store workspace scaffold 2026-06-16 03:48:06 +00:00