- Add QuicSyncBackend + quic_scheduler; fix push stream/close race by
waiting for peer close instead of calling close_and_drain() on the
client side (SyncComplete stream was racing CONNECTION_CLOSE)
- Add PipeWriteHalf::shutdown_push() for client-sends-last QUIC paths;
use it in cmd_push so the server can process SyncComplete before the
connection tears down
- Add SshSyncBackend + ssh_scheduler with subprocess integration tests
- Add clawsync diff command with --porcelain flag and subprocess tests
- Add QUIC subprocess push/pull integration tests
- Fix clippy --tests violations across 5 crates
- Fix broken intra-doc links (reader.rs, lib.rs, scheduler.rs)
- Rewrite crate READMEs; update BENCHMARKS.md with clawsync-fs CDC row
653 tests, 0 failures.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
New commands:
- `clawsync status <dir>` — local A/M/D report vs .clawsync.state cache; no server needed
- `clawsync verify <local> <remote>` — 1-RTT dry-run diff against remote (TCP/QUIC/SSH)
- `--bwlimit <KBPS>` on `sync` — token-bucket throttle on the pipeline writer
SSH transport expanded to all network commands:
- `push` / `pull` — SSH via `clawsync serve <path> --stdio`
- `hdf5-sync` — SSH via `clawsync serve-hdf5 <dir> --stdio`
- New `--stdio` flag on `serve` and `serve-hdf5` for SSH/pipe sessions
- `CLAWSYNC_SSH_COMMAND` env var overrides the ssh binary (enables fake-SSH tests)
- Removed dead `parse_remote` (TCP-only); all commands now use `parse_remote_target`
Bug fix: server-side `println!` in `handle_client_msg` and `handle_hdf5_client_msg`
corrupted the stdio protocol channel when running in `--stdio` mode.
Changed all server diagnostic output to `eprintln!`.
Tests (+16):
- 4 unit + 4 integration tests for `status`
- 3 integration tests for `verify`
- 1 integration test for `--bwlimit`
- 2 fake-SSH pipe tests for `sync` (cold copy + incremental)
- 2 fake-SSH pipe tests for `push` / `pull` (cold copy each)
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Adds a .clawsync.state file in each watched directory that stores
(mtime_ns, size, blake3) per file. On subsequent manifest builds,
files whose nanosecond-precision mtime and size are unchanged reuse
the cached BLAKE3 without reading file content, making re-syncs over
large directories O(changed files) rather than O(total bytes).
- Cache format: tab-separated text, one entry per line (easy to inspect)
- Written atomically via .clawsync.state.tmp rename
- Both cache files excluded from manifests (never transferred to peers)
- Nanosecond mtime catches same-size, sub-second content changes
(avoids the 1-second resolution hazard that trips up rsync)
- 4 new unit tests: cache exclusion, cache hit, cache miss on change,
hex roundtrip
Also:
- Fix clippy: &PathBuf -> &Path in watch_sync_once
- assert_dir_equal in integration tests now skips .clawsync.state and
*.tmp.clawsync files (internal clawsync files, not user data)
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Watch mode (clawsync watch <local> <remote>):
- Performs initial sync then re-syncs on OS FS events (inotify/kqueue/FSEvents)
- Debounces bursts of events with configurable --debounce-ms (default 500)
- Reconnects on each cycle; prints per-sync stats with delta savings %
- --delete, --exclude, --quic flags all supported
- 2 integration tests: initial sync + incremental, new file creation
Integrity fixes:
- FsSyncPullClient: unwrap_or([0;32]) on blake3_map lookup → Protocol error
if server sends FsCdcData for an unrequested path
- FsSyncPullServer: unwrap_or([0;32]) on server manifest lookup → Protocol error
for paths not in server manifest (was unreachable but now explicit)
FsSyncClient API cleanup:
- Remove unused _delete field (push direction deletions are server-controlled)
- Document that delete param is accepted for symmetry but has no effect
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Adds FsSyncPullClient + FsSyncPullServer implementing the reverse
direction of the existing sync protocol. Client sends a FsDirPullRequest
with its local manifest; server diffs against its own tree, sends
FsDirNeed + pipelined FsCdcData (W=16), client reconstructs with atomic
write+rename and mtime preservation, replies with FsFileAck per file.
- New protocol variant FsDirPullRequest (index 22, rkyv append-only)
- FsSyncPullClient and FsSyncPullServer in session.rs with 4 unit tests
- CLI pull-fs subcommand (remote host:port/path, local dir, --delete, --quic)
- 6 integration tests: cold copy, warm no-op, incremental, delete/no-delete, QUIC
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- Add SyncStats.file_bytes: total uncompressed size of transferred files;
lets CLI compute CDC delta savings (e.g. "1.2 MB file bytes, 45 KB wire, 96% savings")
- Update cmd_sync summary to show "X file bytes (Y wire, Z% delta savings)" when
delta savings are non-trivial; falls back to plain "X bytes" for cold copies
- Exclude *.tmp.clawsync from FsManifest::build() unconditionally so leftover
temp files from interrupted syncs never appear in manifests or get transferred
- Add manifest unit test confirming temp file exclusion
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- Add ProgressEvent enum (TotalFiles / FileAcked) to clawsync-fs
- Add FsSyncClient::with_progress() builder accepting an UnboundedSender<ProgressEvent>;
emits one TotalFiles before RTT2 and one FileAcked per completed file
- Add --verbose / -v flag to `clawsync sync`: spawns a background printer
that reports each synced file and byte count to stderr as acks arrive
- Add ServeAllServerQuic harness to serve_all integration tests
- Add 3 QUIC integration tests for serve-all: cold-copy, warm-noop, incremental;
exercises the wait_for_peer_close drain path in handle_any_client
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- Server restores client's modification time (from FsManifestEntry.mtime) after
atomic write+rename, so tools that check mtimes (make, backup software, rsync)
don't see every synced file as freshly written. Best-effort: silently ignored on
FAT32 and read-only filesystems.
- Add session unit test verifying mtime is preserved within ±2s tolerance.
- Document FsCdcRequest/FsCdcNeed variants as deprecated stubs kept only because
rkyv's append-only rule forbids removal; current protocol embeds server chunks
directly in FsDirNeed, eliminating the per-file sub-exchange round-trip.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- protocol: add dry_run: bool to FsDirManifest
- session: FsSyncClient::with_dry_run() sets the flag; run() handles
FsDirDryRun response and returns would_add/modify/remove in SyncStats
with bytes_transferred=0 and no files written
- session (server): if client sends dry_run=true, compute diffs and send
FsDirDryRun instead of FsDirNeed; return without writing anything
- cli: --dry-run flag on `sync`; print +/~/- prefix per path and summary
- tests: 4 new unit tests covering add, modify, unchanged, and
allow_delete+remove scenarios
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
serve-all (clawsync-cli):
- New ServeAll command listens on one port and dispatches on the first
message: IbltRequest/ManifestRequest → onion revision protocol (file path
derived from agent_id: root/<sanitized-id>.claws), Hdf5ManifestRequest →
HDF5 dataset protocol, FsDirManifest → FS general sync.
- Extracted handle_client_msg and handle_hdf5_client_msg inner functions that
take an already-received first SyncMessage; handle_client/handle_hdf5_client
become thin wrappers that call recv() then delegate.
- Added FsSyncServer::handle_with_first_message() for the same pattern on the
FS side (FsSyncServer owns the peer so handle_any_client consumes it there).
- agent_id sanitization strips path-unsafe characters before using as filename.
QuicSyncBackend (clawsync-agent):
- Mirrors TcpSyncBackend exactly but establishes a QuicConnection via
quic_connect(addr, "localhost", QuicConfig::self_signed()) per call.
- Implements push (IBLT pre-flight + pipelined LayerPackets) and pull
(ManifestRequest + packet drain + merge) via the same protocol logic.
- Exported from clawsync_agent as QuicSyncBackend.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Runs cargo fmt --all; all 573 tests still passing, clippy still clean.
No logic changes — formatting only.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
osobh