Adds the wire surface and the cert-loading constructors that
omni-sync's QuicTransport needs for sealed-day replication to a
peer Garage node.
Protocol additions (rkyv discriminants are positional — appended
at the end, never inserted):
- `SyncMessage::SealedFile { peer_name, payload }`: sealed `.h5`
delivery from `omni-sync` clients (omni-cortex daemons
replicating sealed-day Working / Episodic / Semantic tier
files to a peer).
- `SyncMessage::SealedFileAck { peer_name, bytes, blake3_hex }`:
receiver-computed BLAKE3 acknowledgement so the sender can
verify the bytes landed intact.
QuicConfig constructors for production mTLS:
- `client_with_pem(client_cert, client_key, server_ca)` — loads
the operator's mTLS leaf identity + the CA whose-issued server
certs the client trusts. Builds a real rustls `ClientConfig`
via `with_client_auth_cert`. Server config field gets a
placeholder (never consumed by `quic_connect`).
- `server_with_pem(server_cert, server_key, client_ca)` — mirror
for the receive side. Real rustls `ServerConfig` with
`WebPkiClientVerifier::builder(roots).build()` +
`with_client_cert_verifier` (mutual auth: clients without a
CA-issued cert fail the handshake) + `with_single_cert`.
Validated by `omni-sync`'s D193 production-mTLS round-trip test:
rcgen-generates CA + matching server cert + matching client
cert, writes PEMs to a tempdir, spawns a `QuicServer` with
`server_with_pem`, drives a real `QuicTransport` production-mode
client through `SealedFile` → `SealedFileAck` with mutual cert
verification.
Also includes a `rustls-pemfile = "2.2"` dep + ambient
`cargo fmt` line-collapsing in `framed.rs` and `peer.rs` from
when the workspace was reformatted.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
- Add QuicSyncBackend + quic_scheduler; fix push stream/close race by
waiting for peer close instead of calling close_and_drain() on the
client side (SyncComplete stream was racing CONNECTION_CLOSE)
- Add PipeWriteHalf::shutdown_push() for client-sends-last QUIC paths;
use it in cmd_push so the server can process SyncComplete before the
connection tears down
- Add SshSyncBackend + ssh_scheduler with subprocess integration tests
- Add clawsync diff command with --porcelain flag and subprocess tests
- Add QUIC subprocess push/pull integration tests
- Fix clippy --tests violations across 5 crates
- Fix broken intra-doc links (reader.rs, lib.rs, scheduler.rs)
- Rewrite crate READMEs; update BENCHMARKS.md with clawsync-fs CDC row
653 tests, 0 failures.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Adds generic framed I/O transport (FramedReader/FramedWriter over any
AsyncRead/AsyncWrite) and a StreamPeer backed by boxed I/O, enabling
two new modes:
SSH client mode:
clawsync sync <local> [user@]host:path
Detects SSH syntax (@ or non-numeric port), spawns
`ssh user@host clawsync serve-fs path --stdio`, and uses the child's
stdin/stdout as the sync protocol channel. No pre-running server needed.
stdio server mode (--stdio flag on serve-fs / serve-all):
clawsync serve-fs <dir> --stdio
Reads/writes the sync protocol on stdin/stdout instead of binding TCP.
Suppresses the startup message (stdout is the protocol channel).
Used automatically by SSH clients; also usable in scripts/containers.
Transport layer (clawsync-transport):
- New framed.rs: FramedReader<R>, FramedWriter<W>, StreamPeer, split halves
- SyncPeer::Stream variant; into_pipe_halves() splits into StreamReadHalf/WriteHalf
- PipeReadHalf::Stream, PipeWriteHalf::Stream for pipelined W=16 sessions
CLI (clawsync-cli):
- ParsedRemote enum; parse_remote_target() handles both TCP and SSH syntax
- connect_remote() dispatches to TCP/QUIC or ssh_connect()
- cmd_sync and cmd_watch use connect_remote() — SSH works transparently
- cmd_serve_fs and cmd_serve_all gain --stdio flag
- 2 integration tests: SSH (skipped without CLAWSYNC_TEST_SSH=1), stdio no-panic
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Adds FsSyncPullClient + FsSyncPullServer implementing the reverse
direction of the existing sync protocol. Client sends a FsDirPullRequest
with its local manifest; server diffs against its own tree, sends
FsDirNeed + pipelined FsCdcData (W=16), client reconstructs with atomic
write+rename and mtime preservation, replies with FsFileAck per file.
- New protocol variant FsDirPullRequest (index 22, rkyv append-only)
- FsSyncPullClient and FsSyncPullServer in session.rs with 4 unit tests
- CLI pull-fs subcommand (remote host:port/path, local dir, --delete, --quic)
- 6 integration tests: cold copy, warm no-op, incremental, delete/no-delete, QUIC
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- Server restores client's modification time (from FsManifestEntry.mtime) after
atomic write+rename, so tools that check mtimes (make, backup software, rsync)
don't see every synced file as freshly written. Best-effort: silently ignored on
FAT32 and read-only filesystems.
- Add session unit test verifying mtime is preserved within ±2s tolerance.
- Document FsCdcRequest/FsCdcNeed variants as deprecated stubs kept only because
rkyv's append-only rule forbids removal; current protocol embeds server chunks
directly in FsDirNeed, eliminating the per-file sub-exchange round-trip.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- Add QuicConfig::insecure() with NoopVerifier for subprocess integration
tests where server/client generate independent self-signed certs
- Fix QUIC endpoint lifetime bug: store _endpoint in QuicConnection so
quinn's endpoint outlives its connection (previously dropped on return)
- Add install_crypto_provider() to transport lib; call it in main() to
satisfy rustls 0.23's explicit install_default() requirement
- Add QuicConnection::wait_for_peer_close(): server passively awaits
client close instead of racing to send CONNECTION_CLOSE before
in-flight streams (FsDirComplete, FsFileAck) reach the client
- Add SyncPeer::quic_conn_clone() + update shutdown/drain to use
close_and_drain (client) vs wait_for_peer_close (server)
- Add 3 QUIC FS sync integration tests: cold-copy, warm-noop, incremental
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- protocol: add dry_run: bool to FsDirManifest
- session: FsSyncClient::with_dry_run() sets the flag; run() handles
FsDirDryRun response and returns would_add/modify/remove in SyncStats
with bytes_transferred=0 and no files written
- session (server): if client sends dry_run=true, compute diffs and send
FsDirDryRun instead of FsDirNeed; return without writing anything
- cli: --dry-run flag on `sync`; print +/~/- prefix per path and summary
- tests: 4 new unit tests covering add, modify, unchanged, and
allow_delete+remove scenarios
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
MmapSender and MmapReceiver were structural stubs with only open() methods.
Now each holds its own MmapMut view of the channel file and implements full
ring-buffer send/recv using AtomicU64 Acquire/Release stores for correct
cross-process visibility.
Extracted shared ring_send/ring_recv logic used by all three types
(MmapChannel, MmapSender, MmapReceiver). Replaced plain write_u64/read_u64
header accessors with atomic_store/atomic_load in the hot path; plain
helpers retained for one-time initialisation in create().
SyncPeer gains a Mmap { send_ch, receiver } variant; PipeWriteHalf and
PipeReadHalf gain Mmap arms. Async SyncPeer::send/recv/into_pipe_halves
wrap the blocking spin-wait calls with tokio::task::block_in_place.
Added 5 new mmap tests: sender_receiver_open_same_file, roundtrip,
multiple_messages, ring_wraps, cross_thread (two-thread SPSC).
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Runs cargo fmt --all; all 573 tests still passing, clippy still clean.
No logic changes — formatting only.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
osobh