blob: size-based LRU eviction + auto-cap in the GC ticker
Orphan-chunk GC alone doesn't stop unbounded growth: as long as fingerprint→blob refs keep getting PutRef'd, the manifest set keeps growing and no chunk is ever an orphan. * `BlobStore::evict_to_size_cap(max_bytes)` — walks manifests oldest first by mtime, deletes them, refcount-decrements each chunk they used, unlinks + reclaims size for any chunk whose refcount hits zero. Shared chunks stay put until the last blob referencing them is evicted. * `ManifestSummary` internal type keeps the diff-set bookkeeping cheap (one HashMap<ChunkHash, u32>, no repeated tree walks). * `claw-store cluster-gc --evict-to-gb <N>` extends the CLI: still runs the orphan sweep first, then optionally caps the store. * Config: `cluster.blob_max_gb: Option<u64>`. The auto-GC ticker runs eviction after every orphan sweep when this is set. Silent when the store is already under cap; INFO log when it evicts. +3 tests: - evict_to_size_cap_reclaims_oldest_blobs_first: 3 blobs with distinct mtimes, cap below combined size → oldest evicted, newer blobs survive - evict_keeps_shared_chunks_when_still_referenced: guards the refcount decrement path (content-addressed dedup keeps identical content as one blob → chunk survives until manifest deleted) - evict_on_empty_store_is_a_noop: sanity 257 tests pass (baseline +3). Pre-existing macOS failure unchanged.
This commit is contained in:
@@ -161,6 +161,13 @@ pub struct ClusterConfig {
|
||||
/// Typical value: `6` hours on a runner cache.
|
||||
#[serde(default)]
|
||||
pub gc_interval_hours: Option<u64>,
|
||||
/// Field finding 2026-07-12: total blob-store size cap in GiB.
|
||||
/// When set, the auto-GC ticker runs `evict_to_size_cap` after
|
||||
/// its orphan sweep, deleting oldest manifests until the
|
||||
/// live-referenced footprint sits at or below this bound.
|
||||
/// Absent means "grow unbounded".
|
||||
#[serde(default)]
|
||||
pub blob_max_gb: Option<u64>,
|
||||
}
|
||||
|
||||
/// Compute the default RPC address for a gossip address: same IP, port + 1.
|
||||
@@ -420,6 +427,7 @@ tailscale_addr = "100.64.1.5:7701"
|
||||
blob_store_root: None,
|
||||
prom_bind: None,
|
||||
gc_interval_hours: None,
|
||||
blob_max_gb: None,
|
||||
};
|
||||
let err = cluster.validate().unwrap_err().to_string();
|
||||
assert!(err.contains("no bind address"), "unexpected error: {err}");
|
||||
@@ -451,6 +459,7 @@ tailscale_addr = "100.64.1.5:7701"
|
||||
blob_store_root: None,
|
||||
prom_bind: None,
|
||||
gc_interval_hours: None,
|
||||
blob_max_gb: None,
|
||||
};
|
||||
let err = cluster.validate().unwrap_err().to_string();
|
||||
assert!(
|
||||
|
||||
Reference in New Issue
Block a user