# clawsync-fs

CDC-based delta sync for any file type — the general-purpose sync engine behind
`clawsync sync` and `clawsync serve-fs`.

## Overview

`clawsync-fs` closes the gap between ClawSync's HDF5-specific revision sync and
rsync's general-purpose file sync. It handles arbitrary file types and directory
trees using **Content-Defined Chunking** (FastCDC) so chunk boundaries are
insertion-stable: a 1-byte prefix insertion does not invalidate the rest of the
file's chunks.

## Architecture

```
FsSyncClient                          FsSyncServer
  │                                         │
  ├── FsManifest::build (rayon parallel)    │
  │   BLAKE3 every file in local tree       │
  │                                         │
  ├─── FsDirManifest ──────────────────────▶│
  │                                         │ FsManifest::build (server tree)
  │                                         │ diff_manifests → Added/Modified/Removed
  │◀── FsDirNeed ───────────────────────────│
  │    (needed_files with server chunks,    │
  │     to_delete if allow_delete)          │
  │                                         │
  │  [W=16 pipelined — into_pipe_halves()]  │
  ├─── FsCdcData (/path, chunk_order, ─────▶│ reconstruct_file
  │              literal_chunks)            │ atomic write (.tmp → rename)
  │◀── FsFileAck ───────────────────────────│
  │    (repeat for each modified/added      │
  │     file in any order)                  │
  │◀── FsDirComplete ───────────────────────│
```

**Protocol cost:**
| Scenario | RTTs |
|----------|------|
| Warm no-op | 1 |
| Cold copy | 2 |
| Incremental | 2 |

## Modules

### `manifest`

`FsManifest::build(root, excludes)` — parallel BLAKE3 walk of a directory tree
using `walkdir` + Rayon. Returns a sorted manifest of all regular files with
their BLAKE3 hash, size, and mtime.

```rust
let excludes = GlobSet::empty();
let manifest = FsManifest::build(Path::new("./data"), &excludes)?;
for entry in &manifest.entries {
    println!("{} {} bytes", entry.rel_path, entry.size);
}
```

### `differ`

`diff_manifests(local, remote)` — compare two manifest entry lists by path and
BLAKE3 hash. Returns `Vec<FileDiff>` with `Added`, `Modified`, `Removed`, and
`Unchanged` variants.

### `delta`

CDC chunking, transfer-need computation, and file reconstruction.

- `chunk_file_for_request(data)` — chunk bytes and return `Vec<FsChunkHash>`
- `compute_needed_indices(server_existing, client_request)` — which chunks the
  server lacks (by xxHash3-64)
- `build_chunk_data(data, chunks, needed_indices)` — pack literal data for
  transfer; zstd-compresses each chunk, falls back to raw if compressed ≥ raw
- `reconstruct_file(server_file, client_order, server_map, literals, expected_blake3)` —
  rebuild the target file from server-local chunks + received literals; verifies
  BLAKE3 on output

### `session`

`FsSyncClient` and `FsSyncServer` — async protocol orchestration over a
`SyncPeer` (TCP or QUIC).

```rust
// Client
let stats = FsSyncClient::new(peer, local_root, excludes, delete)
    .run()
    .await?;
println!("{} added, {} modified, {} removed", stats.files_added, stats.files_modified, stats.files_removed);

// Server (called per accepted connection)
FsSyncServer::new(peer, serve_root, excludes, allow_delete)
    .handle()
    .await?;
```

## Performance

- **Manifest build:** parallelised with Rayon — scales with core count
- **Chunk transfer:** W=16 pipelined send/recv via `SyncPeer::into_pipe_halves()`
- **CDC boundaries:** FastCDC, immune to insertion staircase that breaks rsync's rolling checksum
- **Compression:** per-chunk zstd; falls back to raw for already-compressed data

## License

MIT — see repository root.