Adds a Rateless Bloom Filter + residual IBLT hybrid for the sync
pre-flight, implementing the divergent-replica reconciliation scheme
from arXiv 2510.27614 (Silva Gomes & Baquero, 2025).
When the local revision count exceeds RBF_REV_COUNT_THRESHOLD (200) —
the heuristic for initial clones and post-partition reconnects — the
client now ships a small RBF + small residual IBLT instead of an
over-provisioned single-stage IBLT. The receiver partitions its key
set by the Bloom filter, builds the candidate-intersection residual
IBLT, and subtracts the client bundle to recover missing_from_remote
in one round trip.
Wire-format change is strictly additive: two new SyncMessage variants
(RbfRequest, RbfResponse) are appended after SealedFileAck; all
existing discriminants are preserved (regression test added). Onion
diff and transport framing are unchanged.
Files modified / added:
- crates/clawsync-onion/src/rbf.rs (new module, 30 unit tests + 3 proptests)
- crates/clawsync-onion/src/manifest.rs (RbfManifest bundle type + 6 tests)
- crates/clawsync-onion/src/lib.rs (re-exports)
- crates/clawsync-transport/src/protocol.rs (RbfRequest/RbfResponse variants + 3 roundtrip tests)
- crates/clawsync-cli/src/main.rs (heuristic switch in client push path + server dispatch)
- crates/clawsync-agent/src/backend.rs (TCP/QUIC/SSH push paths opt into RBF)
- CHANGELOG.md (Unreleased entry)
- crates/clawsync-onion/proptest-regressions/rbf.txt (seed file)
Gates:
- cargo check --workspace: clean
- cargo clippy -p (affected crates) --lib --bins --tests -- -D warnings: clean
- cargo test --workspace: 704 passed / 0 failed (87 in clawsync-onion incl. all RBF tests)
- Add QuicSyncBackend + quic_scheduler; fix push stream/close race by
waiting for peer close instead of calling close_and_drain() on the
client side (SyncComplete stream was racing CONNECTION_CLOSE)
- Add PipeWriteHalf::shutdown_push() for client-sends-last QUIC paths;
use it in cmd_push so the server can process SyncComplete before the
connection tears down
- Add SshSyncBackend + ssh_scheduler with subprocess integration tests
- Add clawsync diff command with --porcelain flag and subprocess tests
- Add QUIC subprocess push/pull integration tests
- Fix clippy --tests violations across 5 crates
- Fix broken intra-doc links (reader.rs, lib.rs, scheduler.rs)
- Rewrite crate READMEs; update BENCHMARKS.md with clawsync-fs CDC row
653 tests, 0 failures.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- differ.rs: build HashMap<u64, RevisionSummary> once in packets_for_revisions
and diff_revisions_merkle instead of O(N) list_revisions() scan per revision
- merger.rs: sort index Vec in verify_packet_hash instead of cloning MB-scale
page data
- backend.rs (Tcp + Quic): eliminate std::fs::read(local_path) — file_blake3
is never validated by server; set to [0u8;32] directly; call list_revisions()
once and reuse; add W=16 pipelined send via Semaphore + into_split() for TCP,
Arc<QuicConnection> clone for QUIC
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Runs cargo fmt --all; all 573 tests still passing, clippy still clean.
No logic changes — formatting only.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Clippy fixes:
- clawhdf5-onion/reader.rs: introduce RawPage type alias to resolve
type_complexity lint on revision_pages_raw return type
- clawsync-onion/iblt.rs: remove stray #[forbid(unsafe_code)] module
attribute (crate-level forbid already in lib.rs; the module attr was
useless and triggered the empty-line-after-attribute lint)
- clawsync-onion/differ.rs: replace ok_or_else(|| …) with ok_or(…)
for non-lazily-constructed error values
- clawsync-cli/main.rs: accept &Path instead of &PathBuf in
handle_hdf5_client to avoid unnecessary indirection
CI (.gitea/workflows/ci.yml):
- test job: cargo test --workspace (default + simd-cdc)
- lint job: cargo clippy -D warnings + cargo fmt --check
- msrv job: build + test on Rust 1.85 (workspace rust-version)
- publish-dry-run job: cargo publish --dry-run for clawsync-core
All 573 tests still passing after fixes.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
rust-refactor