osobhandClaude Sonnet 4.6
6d2eb5a1db
- Add QuicSyncBackend + quic_scheduler; fix push stream/close race by waiting for peer close instead of calling close_and_drain() on the client side (SyncComplete stream was racing CONNECTION_CLOSE) - Add PipeWriteHalf::shutdown_push() for client-sends-last QUIC paths; use it in cmd_push so the server can process SyncComplete before the connection tears down - Add SshSyncBackend + ssh_scheduler with subprocess integration tests - Add clawsync diff command with --porcelain flag and subprocess tests - Add QUIC subprocess push/pull integration tests - Fix clippy --tests violations across 5 crates - Fix broken intra-doc links (reader.rs, lib.rs, scheduler.rs) - Rewrite crate READMEs; update BENCHMARKS.md with clawsync-fs CDC row 653 tests, 0 failures. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2.3 KiB
clawsync-onion
ClawSync onion-aware revision diff, layer packaging, IBLT pre-flight, and manifest protocol.
Overview
clawsync-onion sits between the onion storage layer (clawhdf5-onion) and the
transport layer (clawsync-transport). It knows how to:
- Compute which revisions a remote node is missing (
diff_revisions) - Package those revisions as
OnionLayerPacketwire messages - Apply incoming packets to a local
OnionFile(merge_packets) - Build and exchange
IbltManifestfor sub-linear pre-flight negotiation - Filter revisions by branch, ceiling, or selector (
SyncSelector)
Key types
diff_revisions(local, remote_head) -> Vec<OnionLayerPacket>
Returns the minimal set of packets needed to bring the remote up to date — ascending revision order, ready to send.
merge_packets(onion, packets, flush) -> MergeStats
Applies incoming packets to a local OnionFile. Verifies BLAKE3 integrity,
skips already-present revisions (idempotent), and optionally flushes.
IbltManifest / IbltSketch
Rateless IBLT (Invertible Bloom Lookup Table) for sub-linear pre-flight comparison. At N=1K revisions, IBLT pre-flight requires ~4 ms and ~4 KB of wire data vs ~2 s and ~4 MB for a flat manifest exchange — roughly 500× faster.
// Client side: encode local revisions into a sketch
let sketch = IbltSketch::from_keys(&local_rev_numbers, seed);
let manifest = IbltManifest { sketch: sketch.to_bytes(), .. };
// Server side: compute what the client is missing
let diff = client_manifest.diff_against(&server_revs)?;
// diff.only_in_b = revisions client has that server lacks → send these
ClawSyncManifest
Flat manifest exchanged during pull-direction negotiation. Contains revision count, HEAD revision, and per-revision BLAKE3 hashes. Used when the remote has more data than the local node (pull pre-flight).
SyncSelector
Filter revisions before pushing:
SyncSelector::All // push every revision
SyncSelector::Branch(id) // push only revisions on branch `id`
SyncSelector::UpTo(rev) // push revisions 0..=rev
OnionLayerPacket
Wire representation of a single revision: revision number, branch ID, parent
revision, BLAKE3 root, and a list of OnionPage (h5_offset + page bytes).
Serialized with rkyv for zero-copy deserialization.
License
MIT — see repository root.