Phase 7d follow-on: snapshots pin blobs against LRU eviction
Build with clawstor cache / Cargo build (clawstor-cached) (pull_request) Successful in 10s

Closes the retention loop between snapshots and pin-aware LRU
eviction. A snapshot is not just a "list of blobs at time T" any
more — it's a *retention pin* on every blob it captures.
Operators can guarantee a build stays on disk for N days by
snapshotting it and pruning the snapshot when the window is up.

Additions:
* SnapshotStore::pinned_blob_ids() → union of blob_ids across all
  live snapshots. Cheap: one JSON read per snapshot.
* cmd_cluster_gc extends the tag-pin set with snapshot pins
  before handing it to evict_to_size_cap_with_pins. Output line
  now reads "pinned blobs: N (M from snapshots)".
* ClusterServices auto-GC ticker does the same on every tick;
  log fields include snapshot_pins so ops see the retention set
  size at a glance.

+2 tests:
- pinned_blob_ids_unions_all_snapshots (overlap dedupe)
- pinned_blob_ids_empty_when_no_snapshots

355 tests pass (+2). Pre-existing macOS
hot::tests::test_project_target_size_bytes failure unchanged.
This commit is contained in:
Omar Sobh
2026-07-14 09:30:31 -07:00
parent e564b0ce89
commit eebc62d87b
3 changed files with 83 additions and 3 deletions
+48
View File
@@ -183,6 +183,28 @@ impl SnapshotStore {
}
}
/// Phase 7d follow-on: union of every `blob_id` referenced by ANY
/// snapshot. Feeds pin-aware eviction — every blob captured by a
/// live snapshot survives the LRU cap. Semantically "snapshots
/// act as immortal retention pins until the operator deletes
/// them".
///
/// Cheap: one file read + JSON parse per snapshot. A fleet with
/// 100 snapshots × 10k blobs each is ~1 MB of JSON I/O.
pub async fn pinned_blob_ids(
&self,
) -> Result<std::collections::HashSet<BlobId>> {
let mut out = std::collections::HashSet::new();
for summary in self.list().await? {
if let Some(m) = self.get(&summary.name).await? {
for id in m.blob_ids {
out.insert(id);
}
}
}
Ok(out)
}
fn snapshot_path(&self, name: &str) -> PathBuf {
// Names are validated: no slashes, printable only. Safe to
// use directly as a filename fragment. We still normalize
@@ -347,6 +369,32 @@ mod tests {
assert!(validate_name("v2026.07.14-pre-release").is_ok());
}
#[tokio::test]
async fn pinned_blob_ids_unions_all_snapshots() {
// Two snapshots, some overlap. Union must dedupe.
let (_tmp, blob, snap) = open();
let a = blob.put_bytes(b"pin-alpha").await.unwrap();
let b = blob.put_bytes(b"pin-beta").await.unwrap();
let c = blob.put_bytes(b"pin-gamma").await.unwrap();
// Snapshot 1: captures a, b, c
snap.create("s1", &blob, 1).await.unwrap();
// Snapshot 2 (later): same content, still captures a, b, c
snap.create("s2", &blob, 2).await.unwrap();
let pins = snap.pinned_blob_ids().await.unwrap();
assert_eq!(pins.len(), 3);
assert!(pins.contains(&a));
assert!(pins.contains(&b));
assert!(pins.contains(&c));
}
#[tokio::test]
async fn pinned_blob_ids_empty_when_no_snapshots() {
let (_tmp, blob, snap) = open();
blob.put_bytes(b"blob-with-no-snapshot").await.unwrap();
assert!(snap.pinned_blob_ids().await.unwrap().is_empty());
}
#[tokio::test]
async fn round_trip_preserves_blob_ids_sorted() {
// list_blob_ids order is filesystem-dependent. Snapshot