clawsync/CHANGELOG.md

Changelog

All notable changes to this project are documented in this file.

Format follows Keep a Changelog. Versioning follows Semantic Versioning.

---

[0.1.0] — 2026-04-06

Initial release of the ClawSync workspace.

Added

Core primitives (clawsync-core)

• BLAKE3 + xxHash3-64 checksums with Rayon-parallel batch hashing
• FastCDC content-defined chunking (variable-length, insertion-stable)
• SIMD-accelerated CDC via Gear hash (NEON/SSE2; opt-in simd-cdc feature)
• zstd and lz4 streaming compression wrappers
• Fixed-block delta encoding (copy/insert operations, apply, size accounting)

Wire protocol (clawsync-transport)

• Length-prefixed rkyv framing over TCP, QUIC (quinn/rustls), mmap ring-buffer, and stdin/stdout pipes
• SyncMessage enum with 22 variants (append-only; discriminants stable)
• SyncPeer abstraction unifying TCP / QUIC / Mmap / Stream behind a single send/recv interface
• PipeWriteHalf / PipeReadHalf for W=16 split-pipeline push
• StreamPeer::from_child / from_stdio for SSH and pipe transports
• QUIC with self-signed TLS 1.3 (dev) or custom QuicConfig (production)

ClawOnion VFD (clawhdf5-onion)

• Pure-Rust HDF5 versioning sidecar: diff-based commits, full-state snapshots, branch fork/merge
• zstd + TDT float compression per page (codec negotiable per-file)
• GC policies: KeepLastN, KeepTagged, KeepSince, KeepRevisions + epoch-based O(N) gc()
• Merkle tree over revision layers for O(log N) sync-diff walks
• Rateless IBLT sketches for sub-linear pre-flight revision set comparison
• VersionedFile high-level API: open/create/commit/branch/merge/snapshot/export
• Golden file format freeze test (4417-byte canonical representation)
• revision_page_entries for zero-copy page metadata access

Sync engine (clawsync-onion, clawsync-hdf5)

• IBLT-based push pre-flight: 30× wire savings at N=1K+ revisions vs flat manifest
• W=16 pipelined push: 13–15× faster than stop-and-wait at 1–5 ms simulated RTT
• SyncSelector: filter by branch, revision ceiling, or "all"
• HDF5 dataset-granular sync: only modified datasets transferred per file
• Dataset-level differ, patcher, and manifest with want_delete / allow_delete guards

Agent library (clawsync-agent)

• SyncBackend trait: pluggable push/pull transport
• TcpSyncBackend: push/pull over TCP with W=16 IBLT pipeline
• QuicSyncBackend: push/pull over QUIC (TLS 1.3, self-signed for dev)
• SshSyncBackend: push/pull over SSH pipe — spawns clawsync serve --stdio on demand, no pre-running daemon required; overridable via CLAWSYNC_SSH_COMMAND
• OnionMemory: autonomous flush → sync lifecycle with WAL
• SyncScheduler: debounced background sync with configurable interval

General file sync (clawsync-fs)

• CDC-based delta for any file type: only missing FastCDC chunks transferred
• FsSyncClient / FsSyncServer: 2-RTT push protocol with W=16 pipelining
• FsSyncPullClient / FsSyncPullServer: reverse (server→client) direction
• FsManifest: parallel BLAKE3 walk via rayon; .clawsync.state nanosecond-mtime cache
• FsDiffer: Added/Modified/Removed/Unchanged classification
• FsDelta: chunk-level need computation and reconstruction with BLAKE3 integrity check
• mtime preservation on synced files (±2 s tolerance for FAT/NFS)
• --bwlimit <KBPS> token-bucket rate limiter
• --dry-run / --verbose / --delete / --exclude <glob> options
• fs_status() + FsStatusReport: local-only A/M/D change detection vs .clawsync.state

CLI (clawsync-cli)

• push / pull — revision sync for .onion HDF5 sidecars
• serve — TCP/QUIC server for push/pull; --stdio for SSH pipe
• sync / serve-fs — general file sync (any type, CDC delta)
• pull-fs — reverse file sync (fetch from remote to local)
• hdf5-sync / serve-hdf5 — dataset-granular HDF5 sync
• watch — OS FS event watcher (kqueue/inotify) with debounce + auto-reconnect
• serve-all — single-port universal server dispatching all three protocol types
• list-revisions, rollback, gc, export-revision, snapshot, branch — local onion management
• diff <rev1> [<rev2>] — page-level diff between revisions; --porcelain for machine output
• status — local A/M/D report vs .clawsync.state cache (no network)
• verify — 1-RTT dry-run diff against remote without transferring data
• SSH transport on all network commands: user@host:path syntax spawns remote binary on demand
• --quic flag for QUIC transport on all network commands

Performance (Apple Silicon, cargo bench)

Operation	Measured	Target
Commit 1 page (4 KB)	5.42 µs	—
Reconstruct rev=100 (no snapshot)	68 µs	≤ 1 ms
Reconstruct rev=200 (with snapshot)	8.1 µs	≤ 5 ms
BLAKE3 (Rayon, 1 MB)	10.3 GiB/s	≥ 5 GB/s
Branch fork	3.4 µs	≤ 100 µs
GC 500 revisions	20.6 µs	≤ 2 s
WAN push W=16 vs W=1 @ 1 ms RTT	13.5× faster	—
WAN push W=16 vs W=1 @ 5 ms RTT	14.8× faster	—
Pre-flight (IBLT) vs rsync at 1 GB	~4 ms vs ~2 s	500× faster

Fixed

• QUIC stream/close race on push: PipeWriteHalf::shutdown() called close_and_drain() immediately after the client sent SyncComplete, which could race the in-flight unidirectional QUIC stream before the server read it. Added PipeWriteHalf::shutdown_push() — used by cmd_push — that calls wait_for_peer_close() instead, keeping the connection alive until the server acknowledges receipt and closes its side.

Notes

• MSRV: Rust 1.85 (Rust 2024 edition)
• #![forbid(unsafe_code)] on all clawsync-* crates
• SyncMessage discriminants are stable and append-only; never insert or reorder variants
• Publish blocked on upstream clawhdf5 workspace reaching crates.io; publish order documented in README