Phase 5e: prefetch --pin <tag>
Small, focused extension to Phase 5c's prefetch: an optional
`--pin <tag-name>` flag that skips fingerprint compute entirely
and resolves the tag → BlobId via GetTag, then downloads that.
## Use case
Restore an old cache into a fresh checkout for regression testing:
$ claw-cargo prefetch --pin clawverse:main:2026-07-12
cache HIT — downloading 3221225472 bytes (768 chunks) to /path/target/dev
...
── claw-cargo prefetch ─────────────────────────────
source: --pin clawverse:main:2026-07-12
blob: 8c2f1a…
downloaded: 3221225472 bytes in 12.3s
restored to: /path/target/dev
────────────────────────────────────────────────────
Or diagnose a "why does this build fail against the pinned cache"
question by prefetching the tagged cache and then running cargo
against your current source. Cargo will detect the mismatched
.fingerprint state and rebuild affected crates — that's the point,
you're diffing behaviour between two known-good cache snapshots.
## Changes
- New PrefetchArgs struct (was reusing PeerArgs) with an optional
`pin: Option<String>` field
- resolve_pin(conn, tag) — internal helper that does
GetTag → BlobStat, returning None on either NotFound
- cmd_prefetch branches at the top: --pin → resolve_pin(); default
→ fingerprint-based peer_lookup()
- Rest of the flow is unchanged: BlobStat → BlobGetStream →
restore_target
- Summary output shows `source: --pin <tag>` instead of
`fingerprint: <hex>` when the pinned path was taken
`peer_lookup` (fingerprint path) and `resolve_pin` (tag path) return
the same `Option<(BlobId, BlobStat)>` shape so the downstream code
is identical.
## Live smoke test
`prefetch --help` now advertises --pin with full description.
Missing-tag path prints "no such tag: <name>" and exits 0
(consistent with the fingerprint-miss path).
## Tests (1 new, real QUIC)
- **`end_to_end_tag_resolve_and_stream_restore_over_real_quic`** —
seeds blob store with a 2 MiB "captured target" payload, publishes
a tag pointing at its BlobId, then runs the exact client
sequence `prefetch --pin <tag>` runs internally:
GetTag → BlobStat → BlobGetStream
Verifies bytes reassemble byte-equal to source. Also covers the
missing-tag path.
The pin flow uses the same underlying calls tested separately in
Phase 5b/5c/5d, so the new test proves the composition works rather
than re-verifying primitives.
224 tests pass. Pre-existing macOS-only failure unchanged.
## What's next
- 5f: Gitea webhook pre-fetch — daemon receives PR-open hints and
warms cache for the predicted fingerprint before CI runner starts
- 6: FUSE mount for warm-tier git worktrees so `~/projects/clawverse`
is transparently fleet-shared
- 3: full CRDT metadata layer (only if real conflicts emerge in the
simple tag model)
This commit is contained in:
@@ -976,6 +976,74 @@ async fn list_tags_returns_json_sorted() {
|
||||
assert_eq!(decoded[0].decode_value().unwrap(), [1u8; 32]);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn end_to_end_tag_resolve_and_stream_restore_over_real_quic() {
|
||||
// Phase 5e: the pipeline `claw-cargo prefetch --pin <tag>` runs
|
||||
// internally — put a blob, publish a tag pointing at it, then
|
||||
// GetTag → BlobStat → BlobGetStream to reassemble the content
|
||||
// byte-equal on the other side.
|
||||
let (id_a, id_b) = NodeIdentity::generate_test_pair("a", "b").unwrap();
|
||||
let (_tmp, router) = router_with_full_stack("a", next_port()).await;
|
||||
|
||||
// Seed the blob store with a synthetic "captured target dir".
|
||||
let payload: Vec<u8> = (0..2 * 1024 * 1024).map(|i| (i % 251) as u8).collect();
|
||||
let blob_id = router
|
||||
.blob_store()
|
||||
.unwrap()
|
||||
.put_bytes(&payload)
|
||||
.await
|
||||
.unwrap();
|
||||
// Publish a tag pointing at that blob.
|
||||
router
|
||||
.tag_store()
|
||||
.unwrap()
|
||||
.put("clawverse:main:latest", blob_id.as_bytes())
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let server = QuicServer::bind(loopback(0), id_a).unwrap();
|
||||
let server_addr = server.local_addr().unwrap();
|
||||
let router_srv = router.clone();
|
||||
let accept_task = tokio::spawn(async move {
|
||||
if let Some(Ok(conn)) = server.accept().await {
|
||||
let _ = serve_connection(conn, router_srv).await;
|
||||
}
|
||||
});
|
||||
|
||||
let client = QuicClient::new(loopback(0), id_b).unwrap();
|
||||
let conn = client.connect(server_addr, "a").await.unwrap();
|
||||
|
||||
// Client side of `prefetch --pin <tag>`:
|
||||
// 1. GetTag → BlobId bytes
|
||||
// 2. BlobStat → confirm blob exists + size
|
||||
// 3. BlobGetStream → download into buffer
|
||||
let tag_value = call_get_tag(&conn, "clawverse:main:latest")
|
||||
.await
|
||||
.unwrap()
|
||||
.expect("tag set");
|
||||
assert_eq!(tag_value, *blob_id.as_bytes());
|
||||
let resolved_id = BlobId::from_bytes(tag_value);
|
||||
let stat = call_blob_stat(&conn, &resolved_id).await.unwrap().unwrap();
|
||||
assert_eq!(stat.total_size, payload.len() as u64);
|
||||
|
||||
let mut sink: Vec<u8> = Vec::new();
|
||||
let ok = call_blob_get_stream(&conn, &resolved_id, &mut sink)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(ok);
|
||||
assert_eq!(sink.len(), payload.len());
|
||||
assert_eq!(sink, payload, "reassembled bytes match source");
|
||||
|
||||
// Missing-tag path: prefetch --pin never-set-name reports None.
|
||||
let missing = call_get_tag(&conn, "never-set").await.unwrap();
|
||||
assert!(missing.is_none());
|
||||
|
||||
conn.close(quinn::VarInt::from_u32(0), b"done");
|
||||
client.shutdown().await;
|
||||
tokio::time::sleep(Duration::from_millis(50)).await;
|
||||
accept_task.abort();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn end_to_end_pin_lookup_delete_over_real_quic() {
|
||||
// Full flow: publish a tag → look it up → list → delete → confirm gone.
|
||||
|
||||
Reference in New Issue
Block a user