Commit Graph
7 Commits
Author SHA1 Message Date
osobh bb299f4b77 chore: commit local changes before node reformat
CI / Publish dry-run (clawsync-core) (push) Has been cancelled
CI / Test (stable) (push) Has been cancelled
CI / Clippy + fmt (push) Has been cancelled
CI / MSRV (1.85) (push) Has been cancelled
2026-05-10 13:58:39 -07:00
osobhandClaude Sonnet 4.6 6d2eb5a1db feat: QUIC backend tests, diff command, clippy/doc fixes, QUIC push race fix
- Add QuicSyncBackend + quic_scheduler; fix push stream/close race by
  waiting for peer close instead of calling close_and_drain() on the
  client side (SyncComplete stream was racing CONNECTION_CLOSE)
- Add PipeWriteHalf::shutdown_push() for client-sends-last QUIC paths;
  use it in cmd_push so the server can process SyncComplete before the
  connection tears down
- Add SshSyncBackend + ssh_scheduler with subprocess integration tests
- Add clawsync diff command with --porcelain flag and subprocess tests
- Add QUIC subprocess push/pull integration tests
- Fix clippy --tests violations across 5 crates
- Fix broken intra-doc links (reader.rs, lib.rs, scheduler.rs)
- Rewrite crate READMEs; update BENCHMARKS.md with clawsync-fs CDC row

653 tests, 0 failures.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-04-06 22:40:31 -05:00
osobhandClaude Sonnet 4.6 2ecf894a0a perf: O(1) revision lookups, eliminate full HDF5 read, W=16 agent pipeline
- differ.rs: build HashMap<u64, RevisionSummary> once in packets_for_revisions
  and diff_revisions_merkle instead of O(N) list_revisions() scan per revision
- merger.rs: sort index Vec in verify_packet_hash instead of cloning MB-scale
  page data
- backend.rs (Tcp + Quic): eliminate std::fs::read(local_path) — file_blake3
  is never validated by server; set to [0u8;32] directly; call list_revisions()
  once and reuse; add W=16 pipelined send via Semaphore + into_split() for TCP,
  Arc<QuicConnection> clone for QUIC

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-04-05 08:57:37 -05:00
osobhandClaude Sonnet 4.6 1c107fe58a Apply rustfmt to entire workspace
Runs cargo fmt --all; all 573 tests still passing, clippy still clean.
No logic changes — formatting only.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-04-04 20:31:33 -05:00
osobhandClaude Sonnet 4.6 3d524e2d63 Fix intra-doc link warnings; add clawsync-fs README and metadata
Doc fixes (20 warnings → 1 benign name-collision warning):
- clawsync-transport/quic.rs: QuicConfig::with_cert doesn't exist → backtick
- clawsync-onion/iblt.rs: bare `insert` link → backtick
- clawsync-core/lib.rs: simd_cdc is feature-gated → backtick
- clawhdf5-onion/annotation.rs: RevisionEntry/BranchEntry are in external
  clawhdf5-format crate, not re-exported → backtick
- clawhdf5-onion/gc.rs: compact_dead_epoch_revisions is private; flush links
  broken → backtick
- clawhdf5-onion/provenance.rs: RevisionEntry from external crate → backtick
- clawhdf5-onion/reader.rs: reconstruct_revision/revision_pages → Self:: prefix
- clawhdf5-onion/writer.rs: REV_FLAG_SNAPSHOT → crate::format:: path;
  reconstruct_revision → Self:: prefix
Remaining warning is `format` module/macro name collision — not a broken link.

clawsync-fs crate metadata:
- Add readme, keywords, categories to Cargo.toml
- Write README.md (CDC protocol diagram, module overview, usage examples)

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-04-04 20:29:28 -05:00
osobhandClaude Sonnet 4.6 2e423d2682 Fix clippy warnings and add Gitea CI pipeline
Clippy fixes:
- clawhdf5-onion/reader.rs: introduce RawPage type alias to resolve
  type_complexity lint on revision_pages_raw return type
- clawsync-onion/iblt.rs: remove stray #[forbid(unsafe_code)] module
  attribute (crate-level forbid already in lib.rs; the module attr was
  useless and triggered the empty-line-after-attribute lint)
- clawsync-onion/differ.rs: replace ok_or_else(|| …) with ok_or(…)
  for non-lazily-constructed error values
- clawsync-cli/main.rs: accept &Path instead of &PathBuf in
  handle_hdf5_client to avoid unnecessary indirection

CI (.gitea/workflows/ci.yml):
- test job: cargo test --workspace (default + simd-cdc)
- lint job: cargo clippy -D warnings + cargo fmt --check
- msrv job: build + test on Rust 1.85 (workspace rust-version)
- publish-dry-run job: cargo publish --dry-run for clawsync-core

All 573 tests still passing after fixes.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-04-04 18:59:34 -05:00
osobhandClaude Sonnet 4.6 260e15f5b6 Initial commit: ClawSync v0.1.0
8-crate pure-Rust workspace for revision-aware HDF5 sync.

## Crates
- clawhdf5-onion: ClawOnion VFD — page-level versioned HDF5 storage,
  binary format, writer/reader, branch DAG, GC, snapshots, provenance
- clawsync-core: BLAKE3, xxHash3, FastCDC (+ SIMD NEON), zstd/lz4
- clawsync-onion: IBLT sketch, Merkle tree differ, packet differ/merger,
  ClawSyncManifest, SyncSelector
- clawsync-hdf5: dataset-level manifest, differ, patcher, wire payload
  reconstruction (apply_received_payloads)
- clawsync-transport: TCP, QUIC (quinn 0.11/TLS 1.3), SyncPeer abstraction,
  length-prefixed rkyv wire protocol (21 SyncMessage variants)
- clawsync-agent: OnionMemory, SyncScheduler, TcpSyncBackend,
  PeerCapabilities negotiation
- clawsync-fs: CDC-based delta sync for any file type; FsSyncClient/Server,
  W=16 pipelining, atomic writes
- clawsync-cli: push/pull/serve/hdf5-sync/serve-hdf5/sync/serve-fs +
  all local management commands; --quic on all network commands

## Key features
- IBLT pre-flight: O(revision count) vs rsync's O(file size)
- W=16 sliding-window push: 13–15x speedup over stop-and-wait at WAN RTT
- Dataset-granular HDF5 sync: only modified datasets transferred
- CDC delta for any file type: insertion-stable chunk boundaries
- Full revision DAG: branch, merge, rollback, export, snapshot, GC
- QUIC transport: TLS 1.3, per-message streams via quinn 0.11

## Tests
~573 passing (default features); ~589 with --features simd-cdc

## Performance (Apple Silicon)
- Reconstruct rev=100: 68 µs (target ≤ 1 ms)
- BLAKE3 Rayon 1 MB: 10.3 GiB/s (target ≥ 5 GB/s)
- GC 500 revisions: 20.6 µs (target ≤ 2 s)
- W=16 vs W=1 at 5 ms RTT: 14.8x speedup
- No-op pre-flight at 16 MB: 4 ms vs rsync 35 ms (7.8x)

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-04-04 18:41:22 -05:00