Initial commit: ClawSync v0.1.0

8-crate pure-Rust workspace for revision-aware HDF5 sync.

## Crates
- clawhdf5-onion: ClawOnion VFD — page-level versioned HDF5 storage,
  binary format, writer/reader, branch DAG, GC, snapshots, provenance
- clawsync-core: BLAKE3, xxHash3, FastCDC (+ SIMD NEON), zstd/lz4
- clawsync-onion: IBLT sketch, Merkle tree differ, packet differ/merger,
  ClawSyncManifest, SyncSelector
- clawsync-hdf5: dataset-level manifest, differ, patcher, wire payload
  reconstruction (apply_received_payloads)
- clawsync-transport: TCP, QUIC (quinn 0.11/TLS 1.3), SyncPeer abstraction,
  length-prefixed rkyv wire protocol (21 SyncMessage variants)
- clawsync-agent: OnionMemory, SyncScheduler, TcpSyncBackend,
  PeerCapabilities negotiation
- clawsync-fs: CDC-based delta sync for any file type; FsSyncClient/Server,
  W=16 pipelining, atomic writes
- clawsync-cli: push/pull/serve/hdf5-sync/serve-hdf5/sync/serve-fs +
  all local management commands; --quic on all network commands

## Key features
- IBLT pre-flight: O(revision count) vs rsync's O(file size)
- W=16 sliding-window push: 13–15x speedup over stop-and-wait at WAN RTT
- Dataset-granular HDF5 sync: only modified datasets transferred
- CDC delta for any file type: insertion-stable chunk boundaries
- Full revision DAG: branch, merge, rollback, export, snapshot, GC
- QUIC transport: TLS 1.3, per-message streams via quinn 0.11

## Tests
~573 passing (default features); ~589 with --features simd-cdc

## Performance (Apple Silicon)
- Reconstruct rev=100: 68 µs (target ≤ 1 ms)
- BLAKE3 Rayon 1 MB: 10.3 GiB/s (target ≥ 5 GB/s)
- GC 500 revisions: 20.6 µs (target ≤ 2 s)
- W=16 vs W=1 at 5 ms RTT: 14.8x speedup
- No-op pre-flight at 16 MB: 4 ms vs rsync 35 ms (7.8x)

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
This commit is contained in:
osobh
2026-04-04 18:41:22 -05:00
co-authored by Claude Sonnet 4.6
commit 260e15f5b6
102 changed files with 30098 additions and 0 deletions
+28
View File
@@ -0,0 +1,28 @@
[package]
name = "clawsync-transport"
description = "ClawSync transport backends: QUIC, TCP, mmap"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
readme = "README.md"
keywords = ["sync", "quic", "tcp", "transport", "protocol"]
categories = ["network-programming", "asynchronous"]
[dependencies]
clawsync-core = { workspace = true }
clawsync-onion = { workspace = true }
quinn = { workspace = true }
rustls = { workspace = true }
rcgen = { version = "0.13" }
tokio = { workspace = true }
rkyv = { workspace = true }
thiserror = { workspace = true }
memmap2 = "0.9"
[dev-dependencies]
tempfile = "3"
clawhdf5-onion = { workspace = true }
tokio = { workspace = true }
+55
View File
@@ -0,0 +1,55 @@
# clawsync-transport
ClawSync transport backends: TCP, QUIC, and mmap.
## Overview
`clawsync-transport` provides the network and IPC layer for ClawSync. All
backends share the same `SyncMessage` wire protocol and length-prefixed framing.
## Backends
### TCP (`tcp`)
Async tokio TCP transport. Each `SyncMessage` is framed as a 4-byte LE length
prefix followed by rkyv-serialized body. Suitable for LAN and WAN sync.
```rust
// Server
let server = TcpServer::bind("0.0.0.0:9999".parse()?).await?;
let (mut conn, _addr) = server.accept().await?;
let msg = conn.recv().await?;
// Client
let mut client = TcpConnection::connect("10.0.0.1:9999".parse()?).await?;
client.send(&SyncMessage::ManifestRequest { .. }).await?;
```
### QUIC (`quic`)
Quinn-based QUIC transport with TLS 1.3, 0-RTT, and certificate pinning. One
stream per `LayerPacket`. Preferred for WAN connections with packet loss.
### mmap (`mmap`)
Zero-copy same-node sync via a shared memory-mapped file ring buffer. For
co-located processes on the same machine.
## Protocol
```text
Client Server
│── ManifestRequest ─────────▶│
│◀─ ManifestResponse ─────────│
│── LayerPacket (rev N+1) ───▶│
│◀─ Ack { revision: N+1 } ────│
│ (repeat for each revision)│
│── SyncComplete ─────────────▶│
```
The `SyncMessage` enum covers: `ManifestRequest`, `ManifestResponse`,
`LayerPacket`, `Ack`, `RetryRequest`, `SyncComplete`, `Error`.
## License
MIT — see repository root.
+33
View File
@@ -0,0 +1,33 @@
//! Error types for clawsync-transport.
use thiserror::Error;
#[derive(Debug, Error)]
pub enum TransportError {
#[error("I/O error: {0}")]
Io(#[from] std::io::Error),
#[error("connection closed unexpectedly")]
ConnectionClosed,
#[error("frame too large: {size} bytes (max {max})")]
FrameTooLarge { size: usize, max: usize },
#[error("serialization error: {0}")]
Serialization(String),
#[error("deserialization error: {0}")]
Deserialization(String),
#[error("QUIC error: {0}")]
Quic(String),
#[error("TLS error: {0}")]
Tls(String),
#[error("protocol error: {0}")]
Protocol(String),
#[error("sync onion error: {0}")]
SyncOnion(#[from] clawsync_onion::SyncOnionError),
}
+23
View File
@@ -0,0 +1,23 @@
//! ClawSync transport backends: TCP, QUIC, and mmap.
//!
//! - [`protocol`]: `SyncMessage` wire protocol + length-prefixed framing
//! - [`tcp`]: Async TCP transport (tokio)
//! - [`quic`]: QUIC transport (quinn 0.11, TLS 1.3)
//! - [`mmap`]: Memory-mapped local transport (Phase 4 stub)
//! - [`error`] — [`TransportError`]
#![deny(unsafe_code)]
pub mod error;
pub mod mmap;
pub mod peer;
pub mod protocol;
pub mod quic;
pub mod tcp;
pub use error::TransportError;
pub use mmap::{MmapChannel, DEFAULT_CAPACITY};
pub use peer::{PipeReadHalf, PipeWriteHalf, SyncPeer};
pub use protocol::SyncMessage;
pub use tcp::{TcpConnection, TcpServer};
pub use quic::{QuicConfig, QuicConnection, QuicServer, quic_connect};
+358
View File
@@ -0,0 +1,358 @@
//! Memory-mapped local transport for same-node ClawSync.
// Memory mapping inherently requires unsafe — allowed here only.
#![allow(unsafe_code)]
//!
//! `MmapChannel` implements zero-copy packet exchange between two processes
//! (or two threads) on the same machine using a shared memory-mapped file as
//! a ring buffer.
//!
//! ## Layout of the shared file
//!
//! ```text
//! ┌──────────────────────── Header (64 bytes) ─────────────────────────┐
//! │ magic[8] │ write_head: u64 │ read_head: u64 │ capacity: u64 │
//! │ producer_pid: u32 │ consumer_pid: u32 │ flags: u64 │ reserved[8] │
//! └────────────────────────────────────────────────────────────────────┘
//! ┌─────────────────── Ring-buffer body ───────────────────────────────┐
//! │ [ 4-byte LE length ][ rkyv bytes ] ... │
//! └────────────────────────────────────────────────────────────────────┘
//! ```
//!
//! Messages are framed identically to the TCP transport (4-byte LE length
//! prefix) so the same `SyncMessage` serialization works unchanged.
//!
//! ## Limitations
//! - Single producer / single consumer (no concurrent senders).
//! - Capacity is fixed at creation time.
//! - No cross-machine support (file must be on a shared filesystem).
use std::fs::OpenOptions;
use std::path::{Path, PathBuf};
use memmap2::MmapMut;
use crate::error::TransportError;
use crate::protocol::{SyncMessage, MAX_FRAME_SIZE};
// ─────────────────────────────────────────────────────────────────────────────
// Constants
// ─────────────────────────────────────────────────────────────────────────────
const MAGIC: &[u8; 8] = b"CLAWMMAP";
const HEADER_SIZE: usize = 64;
/// Default ring-buffer capacity: 64 MiB.
pub const DEFAULT_CAPACITY: usize = 64 * 1024 * 1024;
// ─────────────────────────────────────────────────────────────────────────────
// Public API
// ─────────────────────────────────────────────────────────────────────────────
/// A shared memory channel backed by a memory-mapped file.
///
/// Both the sender (`MmapSender`) and receiver (`MmapReceiver`) hold a
/// reference to the same `MmapChannel` (via `Arc` or by opening the same
/// file path).
pub struct MmapChannel {
path: PathBuf,
mmap: MmapMut,
capacity: usize,
}
/// Send half of an `MmapChannel`.
pub struct MmapSender {
_path: PathBuf,
_capacity: usize,
}
/// Receive half of an `MmapChannel`.
pub struct MmapReceiver {
_path: PathBuf,
_capacity: usize,
}
impl MmapChannel {
/// Create a new shared-memory channel file at `path` with `capacity` bytes
/// of ring-buffer space.
pub fn create(path: &Path, capacity: usize) -> Result<Self, TransportError> {
let total = HEADER_SIZE + capacity;
let file = OpenOptions::new()
.read(true)
.write(true)
.create(true)
.truncate(true)
.open(path)
.map_err(TransportError::Io)?;
file.set_len(total as u64).map_err(TransportError::Io)?;
let mut mmap = unsafe { MmapMut::map_mut(&file).map_err(TransportError::Io)? };
// Write header
mmap[..8].copy_from_slice(MAGIC);
write_u64(&mut mmap, 8, 0); // write_head
write_u64(&mut mmap, 16, 0); // read_head
write_u64(&mut mmap, 24, capacity as u64); // capacity
mmap.flush().map_err(TransportError::Io)?;
Ok(Self { path: path.to_owned(), mmap, capacity })
}
/// Open an existing channel file.
pub fn open(path: &Path) -> Result<Self, TransportError> {
let file = OpenOptions::new()
.read(true)
.write(true)
.open(path)
.map_err(TransportError::Io)?;
let mmap = unsafe { MmapMut::map_mut(&file).map_err(TransportError::Io)? };
if &mmap[..8] != MAGIC {
return Err(TransportError::Protocol("invalid mmap channel magic".into()));
}
let capacity = read_u64(&mmap, 24) as usize;
Ok(Self { path: path.to_owned(), mmap, capacity })
}
/// Send a message into the ring buffer.
///
/// Blocks (spin-waits) if there is not enough space yet.
pub fn send(&mut self, msg: &SyncMessage) -> Result<(), TransportError> {
let frame = msg.to_frame().map_err(TransportError::Protocol)?;
let frame_len = frame.len();
if frame_len > self.capacity - 8 {
return Err(TransportError::Protocol(format!(
"message too large for mmap channel: {} bytes",
frame_len
)));
}
// Spin until space is available
loop {
let wh = read_u64(&self.mmap, 8) as usize;
let rh = read_u64(&self.mmap, 16) as usize;
let used = wh.wrapping_sub(rh);
let free = self.capacity.saturating_sub(used);
if free >= frame_len {
break;
}
std::hint::spin_loop();
}
let wh = read_u64(&self.mmap, 8) as usize;
let buf_off = HEADER_SIZE + (wh % self.capacity);
// Write frame (may wrap around)
self.write_ring(buf_off, &frame);
let new_wh = (wh + frame_len) as u64;
write_u64(&mut self.mmap, 8, new_wh);
self.mmap.flush_range(0, 32).map_err(TransportError::Io)?;
Ok(())
}
/// Receive the next message from the ring buffer.
///
/// Blocks (spin-waits) until a complete message is available.
pub fn recv(&mut self) -> Result<SyncMessage, TransportError> {
// Wait for at least 4 bytes (length prefix)
loop {
let wh = read_u64(&self.mmap, 8) as usize;
let rh = read_u64(&self.mmap, 16) as usize;
if wh.wrapping_sub(rh) >= 4 {
break;
}
std::hint::spin_loop();
}
let rh = read_u64(&self.mmap, 16) as usize;
let buf_off = HEADER_SIZE + (rh % self.capacity);
// Read 4-byte length
let mut len_bytes = [0u8; 4];
self.read_ring(buf_off, &mut len_bytes);
let msg_len = u32::from_le_bytes(len_bytes) as usize;
if msg_len > MAX_FRAME_SIZE {
return Err(TransportError::Protocol(format!(
"mmap frame too large: {msg_len}"
)));
}
// Wait for full message
loop {
let wh = read_u64(&self.mmap, 8) as usize;
let rh2 = read_u64(&self.mmap, 16) as usize;
if wh.wrapping_sub(rh2) >= 4 + msg_len {
break;
}
std::hint::spin_loop();
}
let body_off = HEADER_SIZE + ((rh + 4) % self.capacity);
let mut body = vec![0u8; msg_len];
self.read_ring(body_off, &mut body);
// Advance read head
let new_rh = (rh + 4 + msg_len) as u64;
write_u64(&mut self.mmap, 16, new_rh);
self.mmap.flush_range(0, 32).map_err(TransportError::Io)?;
SyncMessage::from_bytes(&body)
.map_err(TransportError::Protocol)
}
/// Path to the backing file.
pub fn path(&self) -> &Path {
&self.path
}
// ── Helpers ──────────────────────────────────────────────────────────────
fn write_ring(&mut self, start: usize, data: &[u8]) {
let cap = self.capacity;
let rel = start - HEADER_SIZE;
for (i, &b) in data.iter().enumerate() {
let off = HEADER_SIZE + (rel + i) % cap;
self.mmap[off] = b;
}
}
fn read_ring(&self, start: usize, buf: &mut [u8]) {
let cap = self.capacity;
let rel = start - HEADER_SIZE;
for (i, slot) in buf.iter_mut().enumerate() {
let off = HEADER_SIZE + (rel + i) % cap;
*slot = self.mmap[off];
}
}
}
// ─────────────────────────────────────────────────────────────────────────────
// Convenience: split into send/receive halves (file-based, for multi-process)
// ─────────────────────────────────────────────────────────────────────────────
impl MmapSender {
pub fn open(path: &Path) -> Result<Self, TransportError> {
let ch = MmapChannel::open(path)?;
Ok(Self { _path: path.to_owned(), _capacity: ch.capacity })
}
}
impl MmapReceiver {
pub fn open(path: &Path) -> Result<Self, TransportError> {
let ch = MmapChannel::open(path)?;
Ok(Self { _path: path.to_owned(), _capacity: ch.capacity })
}
}
// ─────────────────────────────────────────────────────────────────────────────
// Atomic header helpers
// ─────────────────────────────────────────────────────────────────────────────
fn read_u64(mmap: &[u8], off: usize) -> u64 {
let bytes: [u8; 8] = mmap[off..off + 8].try_into().unwrap_or([0u8; 8]);
u64::from_le_bytes(bytes)
}
fn write_u64(mmap: &mut [u8], off: usize, val: u64) {
mmap[off..off + 8].copy_from_slice(&val.to_le_bytes());
}
// ─────────────────────────────────────────────────────────────────────────────
// Tests
// ─────────────────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
use crate::protocol::SyncMessage;
use tempfile::NamedTempFile;
fn tmp_path() -> (NamedTempFile, std::path::PathBuf) {
let f = NamedTempFile::new().unwrap();
let p = f.path().with_extension("mmap");
(f, p)
}
#[test]
fn create_and_open() {
let (_f, path) = tmp_path();
let _ch = MmapChannel::create(&path, DEFAULT_CAPACITY).unwrap();
let _ch2 = MmapChannel::open(&path).unwrap();
}
#[test]
fn send_recv_single_message() {
let (_f, path) = tmp_path();
let mut ch = MmapChannel::create(&path, DEFAULT_CAPACITY).unwrap();
let msg = SyncMessage::Ack { revision: 42 };
ch.send(&msg).unwrap();
let received = ch.recv().unwrap();
assert!(matches!(received, SyncMessage::Ack { revision: 42 }));
}
#[test]
fn send_recv_multiple_messages() {
let (_f, path) = tmp_path();
let mut ch = MmapChannel::create(&path, DEFAULT_CAPACITY).unwrap();
for i in 0u64..5 {
ch.send(&SyncMessage::Ack { revision: i }).unwrap();
}
for i in 0u64..5 {
match ch.recv().unwrap() {
SyncMessage::Ack { revision } => assert_eq!(revision, i),
other => panic!("unexpected: {other:?}"),
}
}
}
#[test]
fn send_recv_error_message() {
let (_f, path) = tmp_path();
let mut ch = MmapChannel::create(&path, DEFAULT_CAPACITY).unwrap();
let msg = SyncMessage::Error { message: "oops".into() };
ch.send(&msg).unwrap();
let recv = ch.recv().unwrap();
assert!(matches!(recv, SyncMessage::Error { .. }));
}
#[test]
fn send_recv_sync_complete() {
let (_f, path) = tmp_path();
let mut ch = MmapChannel::create(&path, DEFAULT_CAPACITY).unwrap();
ch.send(&SyncMessage::SyncComplete {
revisions_transferred: 100,
bytes_transferred: 4096 * 100,
}).unwrap();
match ch.recv().unwrap() {
SyncMessage::SyncComplete { revisions_transferred, .. } => {
assert_eq!(revisions_transferred, 100);
}
other => panic!("unexpected: {other:?}"),
}
}
#[test]
fn ring_wraps_around() {
// Use a tiny capacity to force ring wrap
let cap = 1024;
let (_f, path) = tmp_path();
let mut ch = MmapChannel::create(&path, cap).unwrap();
// Send enough to wrap the ring
for i in 0u64..20 {
ch.send(&SyncMessage::Ack { revision: i }).unwrap();
let recv = ch.recv().unwrap();
assert!(matches!(recv, SyncMessage::Ack { revision } if revision == i));
}
}
#[test]
fn open_invalid_magic_errors() {
let (_f, path) = tmp_path();
std::fs::write(&path, b"BADMAGIC00000000000000000000000000000000").unwrap();
let result = MmapChannel::open(&path);
assert!(result.is_err());
}
}
+115
View File
@@ -0,0 +1,115 @@
//! Unified transport abstraction for sync sessions.
//!
//! Both TCP and QUIC implement the same length-prefixed `SyncMessage` protocol,
//! but their Rust types differ: `TcpConnection` requires `&mut self` for
//! `send`/`recv`, while `QuicConnection` uses `&self` (one stream per message).
//!
//! `SyncPeer` normalises the two so that push/pull/serve/fs-sync logic can be
//! transport-agnostic. For the W=16 pipelined send path, `into_pipe_halves()`
//! splits the peer into independent read and write halves:
//! - **TCP** → `OwnedReadHalf` / `OwnedWriteHalf` via `TcpConnection::into_split()`
//! - **QUIC** → two `Arc<QuicConnection>` clones (both halves share the connection
//! since QUIC `send`/`recv` are `&self`)
use std::sync::Arc;
use crate::error::TransportError;
use crate::protocol::SyncMessage;
use crate::quic::QuicConnection;
use crate::tcp::{TcpConnection, TcpReadHalf, TcpWriteHalf};
// ─────────────────────────────────────────────────────────────────────────────
// SyncPeer
// ─────────────────────────────────────────────────────────────────────────────
/// A unified connection handle for a single sync session (TCP or QUIC).
pub enum SyncPeer {
Tcp(TcpConnection),
Quic(Arc<QuicConnection>),
}
impl SyncPeer {
pub async fn send(&mut self, msg: &SyncMessage) -> Result<(), TransportError> {
match self {
SyncPeer::Tcp(c) => c.send(msg).await,
SyncPeer::Quic(c) => c.send(msg).await,
}
}
pub async fn recv(&mut self) -> Result<SyncMessage, TransportError> {
match self {
SyncPeer::Tcp(c) => c.recv().await,
SyncPeer::Quic(c) => c.recv().await,
}
}
pub async fn shutdown(self) -> Result<(), TransportError> {
match self {
SyncPeer::Tcp(mut c) => c.shutdown().await,
SyncPeer::Quic(c) => {
c.close();
Ok(())
}
}
}
/// Split into independent read/write halves for concurrent pipelined I/O.
///
/// TCP: yields `OwnedReadHalf` / `OwnedWriteHalf` via `into_split()`.
/// QUIC: yields two `Arc` clones — both halves share the connection.
pub fn into_pipe_halves(self) -> (PipeReadHalf, PipeWriteHalf) {
match self {
SyncPeer::Tcp(conn) => {
let (r, w) = conn.into_split();
(PipeReadHalf::Tcp(r), PipeWriteHalf::Tcp(w))
}
SyncPeer::Quic(arc) => {
(PipeReadHalf::Quic(arc.clone()), PipeWriteHalf::Quic(arc))
}
}
}
}
// ─────────────────────────────────────────────────────────────────────────────
// PipeWriteHalf / PipeReadHalf
// ─────────────────────────────────────────────────────────────────────────────
/// Write half for the sliding-window pipeline.
pub enum PipeWriteHalf {
Tcp(TcpWriteHalf),
Quic(Arc<QuicConnection>),
}
impl PipeWriteHalf {
pub async fn send(&mut self, msg: &SyncMessage) -> Result<(), TransportError> {
match self {
PipeWriteHalf::Tcp(h) => h.send(msg).await,
PipeWriteHalf::Quic(c) => c.send(msg).await,
}
}
pub async fn shutdown(self) -> Result<(), TransportError> {
match self {
PipeWriteHalf::Tcp(mut h) => h.shutdown().await,
PipeWriteHalf::Quic(c) => {
c.close();
Ok(())
}
}
}
}
/// Read half for the sliding-window pipeline.
pub enum PipeReadHalf {
Tcp(TcpReadHalf),
Quic(Arc<QuicConnection>),
}
impl PipeReadHalf {
pub async fn recv(&mut self) -> Result<SyncMessage, TransportError> {
match self {
PipeReadHalf::Tcp(h) => h.recv().await,
PipeReadHalf::Quic(c) => c.recv().await,
}
}
}
+658
View File
@@ -0,0 +1,658 @@
//! ClawSync wire protocol: `SyncMessage` enum + framing.
//!
//! Every transport backend (TCP, QUIC, mmap) uses the same `SyncMessage`
//! framing. Messages are serialized with rkyv and framed with a 4-byte
//! little-endian length prefix.
//!
//! ## Session flow
//!
//! **Standard (flat manifest):**
//! ```text
//! Client Server
//! │── ManifestRequest ─────────▶│
//! │◀─ ManifestResponse ─────────│
//! │── LayerPacket (rev N+1) ───▶│
//! │◀─ Ack { revision: N+1 } ────│
//! │ (repeat for each revision)│
//! │◀─ SyncComplete ─────────────│
//! ```
//!
//! **IBLT pre-flight (preferred when both sides support it):**
//! ```text
//! Client Server
//! │── IbltRequest { sketch_a } ──────▶│
//! │◀─ IbltResponse { │
//! │ sketch_b, │
//! │ missing_from_remote, │ ← what client should push
//! │ } ──────────────────────────────│
//! │ client decodes sketch_b │ ← what client should pull
//! │── LayerPacket ... ───────────────▶│
//! │◀─ Ack ... │
//! │◀─ SyncComplete ───────────────────│
//! ```
use rkyv::{Archive, Deserialize, Serialize};
use clawsync_onion::manifest::{ClawSyncManifest, IbltManifest};
use clawsync_onion::packet::OnionLayerPacket;
// ─────────────────────────────────────────────────────────────────────────────
// FS sync wire types
// ─────────────────────────────────────────────────────────────────────────────
/// A CDC chunk descriptor: hash + length, no data bytes.
///
/// Sent in `FsCdcRequest` / `FsDirNeed::FsFileNeed` to let the receiver
/// identify which chunks it already has without receiving the full data.
#[derive(Archive, Deserialize, Serialize, Debug, Clone, PartialEq, Eq)]
pub struct FsChunkHash {
/// xxHash3-64 of the chunk bytes (fast equality check).
pub hash: u64,
/// Byte length of this chunk.
pub length: u32,
}
/// A chunk with literal byte data — the actual transfer payload.
#[derive(Archive, Deserialize, Serialize, Debug, Clone)]
pub struct FsChunk {
/// xxHash3-64, matches the corresponding `FsChunkHash`.
pub hash: u64,
/// Byte data: zstd-compressed when `compressed == true`, raw otherwise.
pub data: Vec<u8>,
/// Whether `data` is zstd-compressed.
pub compressed: bool,
}
/// Per-file entry in a directory manifest.
#[derive(Archive, Deserialize, Serialize, Debug, Clone, PartialEq, Eq)]
pub struct FsManifestEntry {
/// Relative path from the sync root, forward-slash separators.
pub path: String,
/// BLAKE3 digest of the file's current content.
pub blake3: [u8; 32],
/// File size in bytes.
pub size: u64,
/// Last-modified time as Unix seconds (informational; not used for change detection).
pub mtime: i64,
}
/// What the server needs for one particular file.
#[derive(Archive, Deserialize, Serialize, Debug, Clone)]
pub struct FsFileNeed {
/// Relative path of the file.
pub path: String,
/// Server's existing CDC chunk descriptors for this file (empty if the
/// file is new on the server). The client uses these to compute which
/// chunks it must send.
pub server_chunks: Vec<FsChunkHash>,
/// Expected BLAKE3 hash of the client's version (from `FsDirManifest`).
/// Stored here so the server can verify after reconstruction.
pub expected_blake3: [u8; 32],
}
// ─────────────────────────────────────────────────────────────────────────────
// HDF5 sync wire types
// ─────────────────────────────────────────────────────────────────────────────
/// Per-dataset summary sent by the client in `Hdf5ManifestRequest`.
///
/// Contains enough metadata for the server to diff without opening the h5 file.
#[derive(Archive, Deserialize, Serialize, Debug, Clone, PartialEq)]
pub struct Hdf5DatasetEntry {
/// Absolute dataset path within the file, e.g. `/group/embeddings`.
pub path: String,
/// BLAKE3 hash of the raw dataset bytes.
pub blake3: [u8; 32],
/// Raw byte length of the dataset data.
pub byte_len: u64,
/// Dataset shape (dimension sizes).
pub shape: Vec<u64>,
/// `{:?}` format of `clawhdf5::DType`, e.g. `"F64"`, `"F32"`, `"I32"`.
pub dtype: String,
}
/// Maximum allowed frame size (256 MiB). Protects against malformed length
/// fields causing huge allocations.
pub const MAX_FRAME_SIZE: usize = 256 * 1024 * 1024;
/// All messages exchanged between ClawSync peers.
#[derive(Archive, Deserialize, Serialize, Debug, Clone)]
pub enum SyncMessage {
/// Client announces itself and its current HEAD revision.
ManifestRequest {
agent_id: String,
head_revision: u64,
/// Revision count (so the server can compute the delta).
revision_count: u64,
},
/// Server responds with its own manifest.
ManifestResponse { manifest: ClawSyncManifest },
/// One revision packet (sent by either side, depending on push/pull).
LayerPacket { packet: OnionLayerPacket },
/// Acknowledgement of a successfully received + verified revision.
Ack { revision: u64 },
/// Request to retransmit a revision (hash verification failed).
RetryRequest { revision: u64, reason: String },
/// Transfer is complete.
SyncComplete {
revisions_transferred: u64,
bytes_transferred: u64,
},
/// IBLT pre-flight: client sends its revision sketch.
///
/// If the remote supports IBLT it responds with `IbltResponse`; otherwise
/// it responds with `Error` and the client falls back to `ManifestRequest`.
IbltRequest { sketch: IbltManifest },
/// IBLT pre-flight response: server's sketch + decoded missing revisions.
///
/// `missing_from_remote` = revisions the *client* has that the *server*
/// lacks (what to push). The client decodes `sketch` against its own
/// sketch to find revisions it needs to pull.
IbltResponse {
sketch: IbltManifest,
missing_from_remote: Vec<u64>,
},
/// An error occurred; connection will be closed after this message.
Error { message: String },
// ── FS CDC file delta (variants 912) ────────────────────────────────────
//
// IMPORTANT: rkyv assigns discriminants positionally. These variants MUST
// remain appended after `Error` (index 8). Never insert or reorder.
/// Client announces it wants to sync a single file using CDC delta.
///
/// Carries the ordered CDC chunk descriptors of the client's version of
/// the file. Order defines the final file layout on the server.
FsCdcRequest {
path: String,
chunk_hashes: Vec<FsChunkHash>,
},
/// Server replies with the indices (positions in `FsCdcRequest::chunk_hashes`)
/// it needs transferred. Indices are sorted ascending.
FsCdcNeed {
path: String,
needed_indices: Vec<u32>,
},
/// Client sends literal data for exactly the needed chunks.
///
/// `chunks[i]` corresponds to `needed_indices[i]` from `FsCdcNeed`.
///
/// `chunk_order` carries the full ordered chunk layout of the client's
/// file (hash + length per chunk). This lets the server reconstruct
/// without a separate FsCdcRequest/FsCdcNeed sub-exchange in the directory
/// sync path, while remaining self-contained for single-file use too.
/// Wire overhead is ~12 bytes × N_chunks (negligible relative to data).
FsCdcData {
path: String,
/// Full ordered chunk layout of the client's file.
chunk_order: Vec<FsChunkHash>,
/// Literal data for chunks not already on the server.
chunks: Vec<FsChunk>,
},
/// Server confirms it has reconstructed and written the file at `path`.
FsFileAck { path: String },
// ── FS directory tree sync (variants 1316) ───────────────────────────────
/// Client sends a per-file manifest for an entire directory tree.
FsDirManifest {
/// Root path on the client (informational).
local_root: String,
/// Per-file entries; paths are relative to `local_root`.
entries: Vec<FsManifestEntry>,
},
/// Server sends back what it needs after comparing manifests.
FsDirNeed {
/// Files the server needs CDC data for (Added or Modified).
/// Includes the server's own chunk hashes so the client can compute
/// the minimal transfer without an extra round trip.
needed_files: Vec<FsFileNeed>,
/// Paths on the server that are absent from the client's manifest.
/// Only populated when the server was started with `--allow-delete`.
to_delete: Vec<String>,
},
/// Server reports that all changes have been applied.
FsDirComplete {
files_added: u32,
files_modified: u32,
files_removed: u32,
bytes_transferred: u64,
},
/// Dry-run: reports what *would* change without applying anything.
FsDirDryRun {
would_add: Vec<String>,
would_modify: Vec<String>,
would_remove: Vec<String>,
},
// ── HDF5 dataset-granular sync (variants 1721) ───────────────────────────
//
// IMPORTANT: rkyv assigns discriminants positionally. These variants MUST
// remain appended after `FsDirDryRun` (index 16). Never insert or reorder.
/// Client sends its per-dataset manifest and the remote h5 path to sync.
///
/// The server compares this manifest against its own to find which datasets
/// are Added, Modified, or Removed. `remote_path` is relative to the
/// server's serve root (e.g. `"models/weights.h5"`).
Hdf5ManifestRequest {
/// Target path on the server, relative to the serve root.
remote_path: String,
/// BLAKE3 of the client's full h5 file (fast whole-file equality check).
file_blake3: [u8; 32],
/// Per-dataset entries from the client's DatasetManifest.
entries: Vec<Hdf5DatasetEntry>,
/// Whether the client wants datasets absent from its manifest to be
/// removed from the server (requires server `--allow-delete` too).
want_delete: bool,
},
/// Server responds with exactly which datasets it needs and which to delete.
Hdf5ManifestResponse {
/// Paths of datasets the server needs (Added or Modified on client).
datasets_needed: Vec<String>,
/// Paths to remove from the server (only set when `allow_delete` is on).
to_delete: Vec<String>,
},
/// Client sends raw bytes for one dataset (Added or Modified).
///
/// `dtype` is the `{:?}` format of `clawhdf5::DType` (e.g. `"F64"`).
/// `shape` is the dataset's dimension sizes.
Hdf5DataPayload {
path: String,
data: Vec<u8>,
/// HDF5 DType debug string, e.g. `"F64"`, `"F32"`, `"I32"`.
dtype: String,
/// Dataset shape (dimension sizes).
shape: Vec<u64>,
},
/// Server acknowledges successful reconstruction of a dataset.
Hdf5DataAck { path: String },
/// Server reports that the HDF5 sync is complete.
Hdf5SyncComplete {
datasets_added: u32,
datasets_modified: u32,
datasets_removed: u32,
bytes_transferred: u64,
},
}
impl SyncMessage {
/// Serialize to rkyv bytes.
pub fn to_bytes(&self) -> Result<Vec<u8>, String> {
rkyv::to_bytes::<rkyv::rancor::Error>(self)
.map(|v| v.to_vec())
.map_err(|e| e.to_string())
}
/// Deserialize from rkyv bytes.
///
/// Copies into an aligned buffer first so rkyv's alignment requirement is
/// satisfied even when `bytes` is a subslice of a larger buffer.
pub fn from_bytes(bytes: &[u8]) -> Result<Self, String> {
let mut aligned = rkyv::util::AlignedVec::<16>::with_capacity(bytes.len());
aligned.extend_from_slice(bytes);
rkyv::from_bytes::<SyncMessage, rkyv::rancor::Error>(&aligned)
.map_err(|e| e.to_string())
}
/// Encode to a length-prefixed frame: `[u32 LE length][message bytes]`.
pub fn to_frame(&self) -> Result<Vec<u8>, String> {
let body = self.to_bytes()?;
let mut frame = Vec::with_capacity(4 + body.len());
let len = body.len() as u32;
frame.extend_from_slice(&len.to_le_bytes());
frame.extend_from_slice(&body);
Ok(frame)
}
/// Read one length-prefixed message from a byte slice.
///
/// Returns `(message, bytes_consumed)`.
pub fn from_frame(data: &[u8]) -> Result<(Self, usize), String> {
if data.len() < 4 {
return Err("frame too short for length prefix".into());
}
let len = u32::from_le_bytes(data[..4].try_into().unwrap()) as usize;
if len > MAX_FRAME_SIZE {
return Err(format!("frame too large: {len} bytes (max {MAX_FRAME_SIZE})"));
}
if data.len() < 4 + len {
return Err(format!("incomplete frame: have {} bytes, need {}", data.len(), 4 + len));
}
let msg = Self::from_bytes(&data[4..4 + len])?;
Ok((msg, 4 + len))
}
}
// ─────────────────────────────────────────────────────────────────────────────
// Tests
// ─────────────────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
fn ack(revision: u64) -> SyncMessage {
SyncMessage::Ack { revision }
}
fn manifest_request() -> SyncMessage {
SyncMessage::ManifestRequest {
agent_id: "test-agent".to_string(),
head_revision: 5,
revision_count: 6,
}
}
#[test]
fn ack_roundtrip() {
let msg = ack(42);
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
assert!(matches!(recovered, SyncMessage::Ack { revision: 42 }));
}
#[test]
fn manifest_request_roundtrip() {
let msg = manifest_request();
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
match recovered {
SyncMessage::ManifestRequest { agent_id, head_revision, revision_count } => {
assert_eq!(agent_id, "test-agent");
assert_eq!(head_revision, 5);
assert_eq!(revision_count, 6);
}
_ => panic!("wrong variant"),
}
}
#[test]
fn frame_roundtrip() {
let msg = ack(99);
let frame = msg.to_frame().unwrap();
let (recovered, consumed) = SyncMessage::from_frame(&frame).unwrap();
assert_eq!(consumed, frame.len());
assert!(matches!(recovered, SyncMessage::Ack { revision: 99 }));
}
#[test]
fn frame_too_short_errors() {
let result = SyncMessage::from_frame(&[0, 1]);
assert!(result.is_err());
}
#[test]
fn frame_incomplete_body_errors() {
let msg = ack(1);
let mut frame = msg.to_frame().unwrap();
frame.truncate(frame.len() - 1); // drop last byte
let result = SyncMessage::from_frame(&frame);
assert!(result.is_err());
}
#[test]
fn sync_complete_roundtrip() {
let msg = SyncMessage::SyncComplete {
revisions_transferred: 10,
bytes_transferred: 40960,
};
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
match recovered {
SyncMessage::SyncComplete { revisions_transferred, bytes_transferred } => {
assert_eq!(revisions_transferred, 10);
assert_eq!(bytes_transferred, 40960);
}
_ => panic!("wrong variant"),
}
}
#[test]
fn error_message_roundtrip() {
let msg = SyncMessage::Error { message: "something went wrong".to_string() };
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
match recovered {
SyncMessage::Error { message } => assert_eq!(message, "something went wrong"),
_ => panic!("wrong variant"),
}
}
#[test]
fn iblt_request_roundtrip() {
use clawsync_onion::manifest::IbltManifest;
use clawsync_onion::iblt::IbltSketch;
// Build a minimal IbltManifest
let sketch = IbltSketch::from_keys(&[0u64, 1, 2], 0xABCD);
let iblt_manifest = IbltManifest {
agent_id: "test".to_string(),
file_blake3: [0u8; 32],
revision_count: 3,
head_revision: 2,
head_blake3: [0u8; 32],
last_write: 0.0,
sketch: sketch.to_bytes(),
sketch_cells: sketch.cell_count() as u32,
};
let msg = SyncMessage::IbltRequest { sketch: iblt_manifest };
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
assert!(matches!(recovered, SyncMessage::IbltRequest { .. }));
}
#[test]
fn iblt_response_roundtrip() {
use clawsync_onion::manifest::IbltManifest;
use clawsync_onion::iblt::IbltSketch;
let sketch = IbltSketch::from_keys(&[5u64, 6], 0xDEAD);
let iblt_manifest = IbltManifest {
agent_id: "srv".to_string(),
file_blake3: [1u8; 32],
revision_count: 2,
head_revision: 6,
head_blake3: [2u8; 32],
last_write: 1.0,
sketch: sketch.to_bytes(),
sketch_cells: sketch.cell_count() as u32,
};
let msg = SyncMessage::IbltResponse {
sketch: iblt_manifest,
missing_from_remote: vec![7, 8, 9],
};
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
match recovered {
SyncMessage::IbltResponse { missing_from_remote, .. } => {
assert_eq!(missing_from_remote, vec![7, 8, 9]);
}
_ => panic!("wrong variant"),
}
}
// ── FS variant roundtrips ─────────────────────────────────────────────────
#[test]
fn fs_cdc_request_roundtrip() {
let msg = SyncMessage::FsCdcRequest {
path: "data/weights.bin".to_string(),
chunk_hashes: vec![
FsChunkHash { hash: 0xDEAD, length: 65536 },
FsChunkHash { hash: 0xBEEF, length: 32768 },
FsChunkHash { hash: 0xCAFE, length: 65536 },
],
};
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
match recovered {
SyncMessage::FsCdcRequest { path, chunk_hashes } => {
assert_eq!(path, "data/weights.bin");
assert_eq!(chunk_hashes.len(), 3);
assert_eq!(chunk_hashes[1].hash, 0xBEEF);
}
_ => panic!("wrong variant"),
}
}
#[test]
fn fs_cdc_need_roundtrip() {
let msg = SyncMessage::FsCdcNeed {
path: "model.bin".to_string(),
needed_indices: vec![0, 2],
};
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
match recovered {
SyncMessage::FsCdcNeed { path, needed_indices } => {
assert_eq!(path, "model.bin");
assert_eq!(needed_indices, vec![0, 2]);
}
_ => panic!("wrong variant"),
}
}
#[test]
fn fs_cdc_data_roundtrip() {
let msg = SyncMessage::FsCdcData {
path: "cfg.bin".to_string(),
chunk_order: vec![
FsChunkHash { hash: 0x11, length: 65536 },
FsChunkHash { hash: 0x22, length: 65536 },
],
chunks: vec![
FsChunk { hash: 0x11, data: vec![0xAAu8; 100], compressed: true },
FsChunk { hash: 0x22, data: vec![0xBBu8; 200], compressed: false },
],
};
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
match recovered {
SyncMessage::FsCdcData { path, chunk_order, chunks } => {
assert_eq!(path, "cfg.bin");
assert_eq!(chunk_order.len(), 2);
assert_eq!(chunks.len(), 2);
assert!(chunks[0].compressed);
assert!(!chunks[1].compressed);
assert_eq!(chunks[0].data.len(), 100);
}
_ => panic!("wrong variant"),
}
}
#[test]
fn fs_file_ack_roundtrip() {
let msg = SyncMessage::FsFileAck { path: "embeddings.bin".to_string() };
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
assert!(matches!(recovered, SyncMessage::FsFileAck { ref path } if path == "embeddings.bin"));
}
#[test]
fn fs_dir_manifest_roundtrip() {
let msg = SyncMessage::FsDirManifest {
local_root: "/home/user/data".to_string(),
entries: vec![
FsManifestEntry { path: "a.bin".to_string(), blake3: [1u8; 32], size: 4096, mtime: 1000 },
FsManifestEntry { path: "b.bin".to_string(), blake3: [2u8; 32], size: 8192, mtime: 2000 },
],
};
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
match recovered {
SyncMessage::FsDirManifest { entries, .. } => {
assert_eq!(entries.len(), 2);
assert_eq!(entries[0].path, "a.bin");
assert_eq!(entries[1].size, 8192);
}
_ => panic!("wrong variant"),
}
}
#[test]
fn fs_dir_need_roundtrip() {
let msg = SyncMessage::FsDirNeed {
needed_files: vec![
FsFileNeed {
path: "weights.bin".to_string(),
server_chunks: vec![FsChunkHash { hash: 0xAA, length: 65536 }],
expected_blake3: [0xABu8; 32],
},
],
to_delete: vec!["old_model.bin".to_string()],
};
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
match recovered {
SyncMessage::FsDirNeed { needed_files, to_delete } => {
assert_eq!(needed_files.len(), 1);
assert_eq!(needed_files[0].path, "weights.bin");
assert_eq!(to_delete, vec!["old_model.bin"]);
}
_ => panic!("wrong variant"),
}
}
#[test]
fn fs_dir_complete_roundtrip() {
let msg = SyncMessage::FsDirComplete {
files_added: 2, files_modified: 3, files_removed: 1, bytes_transferred: 1_048_576,
};
let bytes = msg.to_bytes().unwrap();
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
match recovered {
SyncMessage::FsDirComplete { files_added, bytes_transferred, .. } => {
assert_eq!(files_added, 2);
assert_eq!(bytes_transferred, 1_048_576);
}
_ => panic!("wrong variant"),
}
}
/// Regression guard: ManifestRequest must remain at discriminant 0.
/// If rkyv variant ordering ever changes, this catches it immediately.
#[test]
fn manifest_request_discriminant_unchanged() {
let msg = manifest_request();
let bytes = msg.to_bytes().unwrap();
// rkyv 0.8 places the discriminant as the last 4 bytes of the archive.
// We just verify the roundtrip still produces the exact same ManifestRequest —
// any reordering would cause a deserialization error here.
let recovered = SyncMessage::from_bytes(&bytes).unwrap();
assert!(matches!(recovered, SyncMessage::ManifestRequest { .. }),
"ManifestRequest discriminant changed — rkyv variant ordering was modified");
}
#[test]
fn multiple_frames_in_buffer() {
// Pack two frames together, verify we can read them sequentially
let msg1 = ack(1);
let msg2 = ack(2);
let mut buf = msg1.to_frame().unwrap();
buf.extend(msg2.to_frame().unwrap());
let (r1, n1) = SyncMessage::from_frame(&buf).unwrap();
let (r2, n2) = SyncMessage::from_frame(&buf[n1..]).unwrap();
assert!(matches!(r1, SyncMessage::Ack { revision: 1 }));
assert!(matches!(r2, SyncMessage::Ack { revision: 2 }));
assert_eq!(n1 + n2, buf.len());
}
}
+246
View File
@@ -0,0 +1,246 @@
//! QUIC transport backend (Phase 2 scaffold).
//!
//! Uses `quinn 0.11.x` with rustls TLS 1.3. Each `SyncMessage` is sent
//! as a framed body on a new unidirectional QUIC stream (reliable, ordered
//! per-stream delivery).
//!
//! # Configuration
//!
//! For development / testing, use [`QuicConfig::self_signed`] which generates
//! a temporary self-signed certificate. Production deployments should use
//! [`QuicConfig::with_cert`] with a properly-signed certificate.
//!
//! # Status
//!
//! Phase 2 scaffold: connection setup and send/recv are implemented.
//! 0-RTT and cert pinning are Phase 3 deliverables.
use std::net::SocketAddr;
use std::sync::Arc;
use quinn::{ClientConfig, Connection, Endpoint, ServerConfig};
use rustls::pki_types::{CertificateDer, PrivateKeyDer};
use crate::error::TransportError;
use crate::protocol::SyncMessage;
// ─────────────────────────────────────────────────────────────────────────────
// QuicConfig
// ─────────────────────────────────────────────────────────────────────────────
/// QUIC endpoint configuration.
pub struct QuicConfig {
pub server_config: ServerConfig,
pub client_config: ClientConfig,
}
impl QuicConfig {
/// Build a config with a self-signed certificate.
///
/// The certificate is valid for `localhost` / `127.0.0.1`. The returned
/// `client_config` trusts only this certificate (cert pinning for testing).
pub fn self_signed() -> Result<Self, TransportError> {
// Generate self-signed cert with rcgen
let cert = rcgen::generate_simple_self_signed(vec!["localhost".to_string()])
.map_err(|e| TransportError::Tls(e.to_string()))?;
let cert_der = CertificateDer::from(cert.cert.der().to_vec());
let key_der = PrivateKeyDer::try_from(cert.key_pair.serialize_der())
.map_err(|e| TransportError::Tls(e.to_string()))?;
// Server config
let server_config = ServerConfig::with_single_cert(
vec![cert_der.clone()],
key_der,
)
.map_err(|e| TransportError::Tls(e.to_string()))?;
// Client config: trust only our self-signed cert
let mut roots = rustls::RootCertStore::empty();
roots
.add(cert_der)
.map_err(|e| TransportError::Tls(e.to_string()))?;
let client_config = ClientConfig::with_root_certificates(Arc::new(roots))
.map_err(|e| TransportError::Tls(e.to_string()))?;
Ok(Self { server_config, client_config })
}
}
// ─────────────────────────────────────────────────────────────────────────────
// QuicConnection
// ─────────────────────────────────────────────────────────────────────────────
/// A QUIC connection capable of sending/receiving `SyncMessage`s.
pub struct QuicConnection {
conn: Connection,
}
impl QuicConnection {
pub fn new(conn: Connection) -> Self {
Self { conn }
}
/// Send a `SyncMessage` on a new unidirectional QUIC stream.
pub async fn send(&self, msg: &SyncMessage) -> Result<(), TransportError> {
let body = msg
.to_bytes()
.map_err(TransportError::Serialization)?;
let mut send = self
.conn
.open_uni()
.await
.map_err(|e| TransportError::Quic(e.to_string()))?;
// Length-prefix
let len = body.len() as u32;
send.write_all(&len.to_le_bytes())
.await
.map_err(|e| TransportError::Io(e.into()))?;
send.write_all(&body)
.await
.map_err(|e| TransportError::Io(e.into()))?;
send.finish().map_err(|e| TransportError::Quic(e.to_string()))?;
Ok(())
}
/// Receive one `SyncMessage` from the next incoming unidirectional stream.
pub async fn recv(&self) -> Result<SyncMessage, TransportError> {
let mut recv = self
.conn
.accept_uni()
.await
.map_err(|e| TransportError::Quic(e.to_string()))?;
// Length prefix
let mut len_buf = [0u8; 4];
recv.read_exact(&mut len_buf)
.await
.map_err(|e| TransportError::Quic(e.to_string()))?;
let len = u32::from_le_bytes(len_buf) as usize;
let mut body = vec![0u8; len];
recv.read_exact(&mut body)
.await
.map_err(|e| TransportError::Quic(e.to_string()))?;
SyncMessage::from_bytes(&body).map_err(TransportError::Deserialization)
}
/// Close the QUIC connection with a normal code.
pub fn close(&self) {
self.conn.close(0u32.into(), b"done");
}
}
// ─────────────────────────────────────────────────────────────────────────────
// QuicServer
// ─────────────────────────────────────────────────────────────────────────────
/// A QUIC server endpoint.
pub struct QuicServer {
endpoint: Endpoint,
pub local_addr: SocketAddr,
}
impl QuicServer {
/// Bind a QUIC server endpoint.
pub async fn bind(addr: SocketAddr, config: QuicConfig) -> Result<Self, TransportError> {
let endpoint = Endpoint::server(config.server_config, addr)
.map_err(TransportError::Io)?;
let local_addr = endpoint.local_addr()?;
Ok(Self { endpoint, local_addr })
}
/// Accept the next incoming QUIC connection.
pub async fn accept(&self) -> Result<QuicConnection, TransportError> {
let incoming = self
.endpoint
.accept()
.await
.ok_or(TransportError::ConnectionClosed)?;
let conn = incoming
.await
.map_err(|e| TransportError::Quic(e.to_string()))?;
Ok(QuicConnection::new(conn))
}
}
// ─────────────────────────────────────────────────────────────────────────────
// QuicClient
// ─────────────────────────────────────────────────────────────────────────────
/// Connect to a QUIC server.
pub async fn quic_connect(
server_addr: SocketAddr,
server_name: &str,
config: QuicConfig,
) -> Result<QuicConnection, TransportError> {
let mut endpoint = Endpoint::client("0.0.0.0:0".parse().unwrap())
.map_err(TransportError::Io)?;
endpoint.set_default_client_config(config.client_config);
let conn = endpoint
.connect(server_addr, server_name)
.map_err(|e| TransportError::Quic(e.to_string()))?
.await
.map_err(|e| TransportError::Quic(e.to_string()))?;
Ok(QuicConnection::new(conn))
}
// ─────────────────────────────────────────────────────────────────────────────
// Tests
// ─────────────────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
use std::net::{IpAddr, Ipv4Addr};
fn any_addr() -> SocketAddr {
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 0)
}
#[tokio::test]
async fn quic_ack_roundtrip() {
let config = QuicConfig::self_signed().unwrap();
let server_config = config.server_config;
// Rebuild client config from a fresh self-signed to get separate cert
let cert = rcgen::generate_simple_self_signed(vec!["localhost".to_string()]).unwrap();
let cert_der = CertificateDer::from(cert.cert.der().to_vec());
let key_der = PrivateKeyDer::try_from(cert.key_pair.serialize_der()).unwrap();
let srv_cfg = ServerConfig::with_single_cert(vec![cert_der.clone()], key_der).unwrap();
let mut roots = rustls::RootCertStore::empty();
roots.add(cert_der).unwrap();
let cli_cfg = ClientConfig::with_root_certificates(Arc::new(roots)).unwrap();
let server = QuicServer::bind(any_addr(), QuicConfig {
server_config: srv_cfg,
client_config: cli_cfg.clone(),
}).await.unwrap();
let addr = server.local_addr;
let server_task = tokio::spawn(async move {
let conn = server.accept().await.unwrap();
let msg = conn.recv().await.unwrap();
assert!(matches!(msg, SyncMessage::Ack { revision: 7 }));
conn.send(&SyncMessage::Ack { revision: 7 }).await.unwrap();
tokio::time::sleep(tokio::time::Duration::from_millis(50)).await;
});
let _ = server_config; // suppress unused warning
let mut endpoint = Endpoint::client("0.0.0.0:0".parse().unwrap()).unwrap();
endpoint.set_default_client_config(cli_cfg);
let conn = endpoint.connect(addr, "localhost").unwrap().await.unwrap();
let qconn = QuicConnection::new(conn);
qconn.send(&SyncMessage::Ack { revision: 7 }).await.unwrap();
let reply = qconn.recv().await.unwrap();
assert!(matches!(reply, SyncMessage::Ack { revision: 7 }));
server_task.await.unwrap();
}
}
+274
View File
@@ -0,0 +1,274 @@
//! TCP transport backend.
//!
//! Provides synchronous (blocking) and async (tokio) TCP framed transport.
//! Each message is sent as a 4-byte LE length prefix followed by the rkyv
//! serialized `SyncMessage` body.
use std::net::SocketAddr;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
use tokio::net::tcp::{OwnedReadHalf, OwnedWriteHalf};
use crate::error::TransportError;
use crate::protocol::{SyncMessage, MAX_FRAME_SIZE};
// ─────────────────────────────────────────────────────────────────────────────
// TcpConnection — async read/write of SyncMessages
// ─────────────────────────────────────────────────────────────────────────────
/// An established async TCP connection capable of sending/receiving
/// `SyncMessage` frames.
pub struct TcpConnection {
stream: TcpStream,
}
impl TcpConnection {
/// Wrap an existing `TcpStream`.
pub fn new(stream: TcpStream) -> Self {
Self { stream }
}
/// Connect to `addr` and return a `TcpConnection`.
pub async fn connect(addr: SocketAddr) -> Result<Self, TransportError> {
let stream = TcpStream::connect(addr).await?;
Ok(Self { stream })
}
/// Send a `SyncMessage` over the connection.
pub async fn send(&mut self, msg: &SyncMessage) -> Result<(), TransportError> {
let body = msg
.to_bytes()
.map_err(TransportError::Serialization)?;
let len = body.len() as u32;
self.stream.write_all(&len.to_le_bytes()).await?;
self.stream.write_all(&body).await?;
self.stream.flush().await?;
Ok(())
}
/// Receive one `SyncMessage` from the connection.
pub async fn recv(&mut self) -> Result<SyncMessage, TransportError> {
// Read 4-byte length prefix
let mut len_buf = [0u8; 4];
self.stream.read_exact(&mut len_buf).await?;
let len = u32::from_le_bytes(len_buf) as usize;
if len > MAX_FRAME_SIZE {
return Err(TransportError::FrameTooLarge {
size: len,
max: MAX_FRAME_SIZE,
});
}
// Read body
let mut body = vec![0u8; len];
self.stream.read_exact(&mut body).await?;
SyncMessage::from_bytes(&body)
.map_err(TransportError::Deserialization)
}
/// Gracefully shut down the write half of the connection.
pub async fn shutdown(&mut self) -> Result<(), TransportError> {
self.stream.shutdown().await?;
Ok(())
}
/// Consume this connection and split into independent read and write halves.
///
/// The halves can be used concurrently from separate tasks, enabling
/// pipelined send/receive (sliding window) without holding a single lock.
pub fn into_split(self) -> (TcpReadHalf, TcpWriteHalf) {
let (read_half, write_half) = self.stream.into_split();
(TcpReadHalf { half: read_half }, TcpWriteHalf { half: write_half })
}
}
// ─────────────────────────────────────────────────────────────────────────────
// Split halves — independent read and write ends of a TcpConnection
// ─────────────────────────────────────────────────────────────────────────────
/// The read end of a split `TcpConnection`.
pub struct TcpReadHalf {
half: OwnedReadHalf,
}
impl TcpReadHalf {
/// Receive one `SyncMessage` from the read half.
pub async fn recv(&mut self) -> Result<SyncMessage, TransportError> {
let mut len_buf = [0u8; 4];
self.half.read_exact(&mut len_buf).await?;
let len = u32::from_le_bytes(len_buf) as usize;
if len > MAX_FRAME_SIZE {
return Err(TransportError::FrameTooLarge { size: len, max: MAX_FRAME_SIZE });
}
let mut body = vec![0u8; len];
self.half.read_exact(&mut body).await?;
SyncMessage::from_bytes(&body).map_err(TransportError::Deserialization)
}
}
/// The write end of a split `TcpConnection`.
pub struct TcpWriteHalf {
half: OwnedWriteHalf,
}
impl TcpWriteHalf {
/// Send a `SyncMessage` through the write half.
pub async fn send(&mut self, msg: &SyncMessage) -> Result<(), TransportError> {
let body = msg.to_bytes().map_err(TransportError::Serialization)?;
let len = body.len() as u32;
self.half.write_all(&len.to_le_bytes()).await?;
self.half.write_all(&body).await?;
self.half.flush().await?;
Ok(())
}
/// Gracefully shut down the write half.
pub async fn shutdown(&mut self) -> Result<(), TransportError> {
self.half.shutdown().await?;
Ok(())
}
}
// ─────────────────────────────────────────────────────────────────────────────
// TcpServer — accept loop
// ─────────────────────────────────────────────────────────────────────────────
/// A listening TCP server that accepts `TcpConnection`s.
pub struct TcpServer {
listener: TcpListener,
pub local_addr: SocketAddr,
}
impl TcpServer {
/// Bind to `addr` and start listening.
pub async fn bind(addr: SocketAddr) -> Result<Self, TransportError> {
let listener = TcpListener::bind(addr).await?;
let local_addr = listener.local_addr()?;
Ok(Self { listener, local_addr })
}
/// Accept the next incoming connection.
pub async fn accept(&self) -> Result<(TcpConnection, SocketAddr), TransportError> {
let (stream, peer_addr) = self.listener.accept().await?;
Ok((TcpConnection::new(stream), peer_addr))
}
}
// ─────────────────────────────────────────────────────────────────────────────
// Tests
// ─────────────────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
use std::net::{IpAddr, Ipv4Addr};
fn any_addr() -> SocketAddr {
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 0)
}
#[tokio::test]
async fn connect_and_send_ack() {
let server = TcpServer::bind(any_addr()).await.unwrap();
let addr = server.local_addr;
let server_task = tokio::spawn(async move {
let (mut conn, _) = server.accept().await.unwrap();
let msg = conn.recv().await.unwrap();
assert!(matches!(msg, SyncMessage::Ack { revision: 42 }));
// Echo it back
conn.send(&msg).await.unwrap();
});
let mut client = TcpConnection::connect(addr).await.unwrap();
client.send(&SyncMessage::Ack { revision: 42 }).await.unwrap();
let reply = client.recv().await.unwrap();
assert!(matches!(reply, SyncMessage::Ack { revision: 42 }));
server_task.await.unwrap();
}
#[tokio::test]
async fn send_multiple_messages() {
let server = TcpServer::bind(any_addr()).await.unwrap();
let addr = server.local_addr;
let server_task = tokio::spawn(async move {
let (mut conn, _) = server.accept().await.unwrap();
for expected_rev in 0u64..5 {
let msg = conn.recv().await.unwrap();
assert!(matches!(msg, SyncMessage::Ack { revision } if revision == expected_rev));
}
});
let mut client = TcpConnection::connect(addr).await.unwrap();
for rev in 0u64..5 {
client.send(&SyncMessage::Ack { revision: rev }).await.unwrap();
}
server_task.await.unwrap();
}
#[tokio::test]
async fn sync_complete_roundtrip() {
let server = TcpServer::bind(any_addr()).await.unwrap();
let addr = server.local_addr;
let server_task = tokio::spawn(async move {
let (mut conn, _) = server.accept().await.unwrap();
let msg = conn.recv().await.unwrap();
conn.send(&msg).await.unwrap();
});
let msg = SyncMessage::SyncComplete {
revisions_transferred: 7,
bytes_transferred: 28672,
};
let mut client = TcpConnection::connect(addr).await.unwrap();
client.send(&msg).await.unwrap();
let reply = client.recv().await.unwrap();
match reply {
SyncMessage::SyncComplete { revisions_transferred, bytes_transferred } => {
assert_eq!(revisions_transferred, 7);
assert_eq!(bytes_transferred, 28672);
}
_ => panic!("wrong variant"),
}
server_task.await.unwrap();
}
#[tokio::test]
async fn manifest_request_roundtrip() {
let server = TcpServer::bind(any_addr()).await.unwrap();
let addr = server.local_addr;
let server_task = tokio::spawn(async move {
let (mut conn, _) = server.accept().await.unwrap();
let msg = conn.recv().await.unwrap();
conn.send(&msg).await.unwrap();
});
let msg = SyncMessage::ManifestRequest {
agent_id: "my-agent".to_string(),
head_revision: 3,
revision_count: 4,
};
let mut client = TcpConnection::connect(addr).await.unwrap();
client.send(&msg).await.unwrap();
let reply = client.recv().await.unwrap();
match reply {
SyncMessage::ManifestRequest { agent_id, head_revision, .. } => {
assert_eq!(agent_id, "my-agent");
assert_eq!(head_revision, 3);
}
_ => panic!("wrong variant"),
}
server_task.await.unwrap();
}
}
@@ -0,0 +1,418 @@
//! End-to-end QUIC sync integration test.
//!
//! Mirrors `e2e_sync.rs` but uses `QuicServer` / `QuicConnection` /
//! `quic_connect` instead of the TCP transport. The same direction-aware
//! push protocol applies; only the framing layer changes.
use std::net::{IpAddr, Ipv4Addr, SocketAddr};
use std::sync::Arc;
use clawhdf5_onion::format::NO_PARENT;
use clawhdf5_onion::writer::OnionFile;
use clawsync_onion::{differ::diff_revisions, manifest::ClawSyncManifest, merger::merge_packets};
use clawsync_transport::{
protocol::SyncMessage,
quic::{quic_connect, QuicConfig, QuicServer},
};
use tempfile::NamedTempFile;
fn localhost_any() -> SocketAddr {
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 0)
}
const H5_BASE: &[u8] = b"\x89HDF\r\n\x1a\n";
// ─────────────────────────────────────────────────────────────────────────────
// Helpers
// ─────────────────────────────────────────────────────────────────────────────
/// Create an `OnionFile` with `n` committed revisions.
fn make_onion(n: u8) -> (NamedTempFile, OnionFile) {
let tmp = NamedTempFile::new().unwrap();
let h5_path = tmp.path().with_extension("h5");
std::fs::write(&h5_path, H5_BASE).unwrap();
let mut onion = OnionFile::create(&h5_path, 4096).unwrap();
for i in 0..n {
let mut s = onion.begin_session(None).unwrap();
s.record_page(0, &vec![i; 4096]);
onion.commit_session(s, Some(&format!("rev {i}"))).unwrap();
}
(tmp, onion)
}
/// Build a pair of self-signed QuicConfigs. The server config uses one cert;
/// the client config trusts that cert. The client's QuicConfig also needs a
/// valid (but unused) server_config, so we generate a separate cert for it.
fn make_quic_configs() -> (QuicConfig, QuicConfig) {
fn gen_cert() -> (
rustls::pki_types::CertificateDer<'static>,
rustls::pki_types::PrivateKeyDer<'static>,
) {
let cert = rcgen::generate_simple_self_signed(vec!["localhost".to_string()]).unwrap();
let cert_der = rustls::pki_types::CertificateDer::from(cert.cert.der().to_vec());
let key_der =
rustls::pki_types::PrivateKeyDer::try_from(cert.key_pair.serialize_der()).unwrap();
(cert_der, key_der)
}
// Server cert: used for TLS handshake; client must trust it.
let (srv_cert_der, srv_key_der) = gen_cert();
let srv_server_config =
quinn::ServerConfig::with_single_cert(vec![srv_cert_der.clone()], srv_key_der).unwrap();
let mut roots = rustls::RootCertStore::empty();
roots.add(srv_cert_der).unwrap();
let client_config =
quinn::ClientConfig::with_root_certificates(Arc::new(roots)).unwrap();
// Client-side QuicConfig requires a server_config field; generate a fresh
// cert+key pair so there is no key/cert mismatch.
let (cli_cert_der, cli_key_der) = gen_cert();
let cli_server_config =
quinn::ServerConfig::with_single_cert(vec![cli_cert_der], cli_key_der).unwrap();
let srv_cfg = QuicConfig { server_config: srv_server_config, client_config: client_config.clone() };
let cli_cfg = QuicConfig { server_config: cli_server_config, client_config };
(srv_cfg, cli_cfg)
}
// ─────────────────────────────────────────────────────────────────────────────
// Core helper: run a full push from `src` → `dst` over QUIC loopback.
// Returns (packets_transferred, final_dst).
// ─────────────────────────────────────────────────────────────────────────────
async fn run_quic_push(src: OnionFile, mut dst: OnionFile) -> (u64, OnionFile) {
let (srv_cfg, cli_cfg) = make_quic_configs();
let server = QuicServer::bind(localhost_any(), srv_cfg).await.unwrap();
let server_addr = server.local_addr;
let src_manifest = ClawSyncManifest::from_onion("src", &src, H5_BASE);
// ── Server task ──────────────────────────────────────────────────────────
let server_handle = tokio::spawn(async move {
let conn = server.accept().await.unwrap();
let mut packets_received: u64 = 0;
loop {
let msg = conn.recv().await.unwrap();
match msg {
SyncMessage::ManifestRequest { .. } => {
let dst_manifest = ClawSyncManifest::from_onion("dst", &dst, H5_BASE);
conn.send(&SyncMessage::ManifestResponse { manifest: dst_manifest })
.await
.unwrap();
}
SyncMessage::LayerPacket { packet } => {
let rev = packet.revision;
merge_packets(&mut dst, vec![packet], false).unwrap();
packets_received += 1;
conn.send(&SyncMessage::Ack { revision: rev }).await.unwrap();
}
SyncMessage::SyncComplete { .. } => {
dst.flush().unwrap();
conn.close();
break;
}
other => panic!("server: unexpected {other:?}"),
}
}
(packets_received, dst)
});
// ── Client logic ─────────────────────────────────────────────────────────
let client = quic_connect(server_addr, "localhost", cli_cfg).await.unwrap();
client
.send(&SyncMessage::ManifestRequest {
agent_id: "src".to_string(),
head_revision: src_manifest.head_revision,
revision_count: src_manifest.revision_count,
})
.await
.unwrap();
let remote_head = match client.recv().await.unwrap() {
SyncMessage::ManifestResponse { manifest } => {
if manifest.revision_count == 0 {
NO_PARENT
} else {
manifest.head_revision
}
}
other => panic!("client: expected ManifestResponse, got {other:?}"),
};
let packets = diff_revisions(&src, remote_head).unwrap();
let total_sent = packets.len() as u64;
for packet in packets {
let rev = packet.revision;
client.send(&SyncMessage::LayerPacket { packet }).await.unwrap();
match client.recv().await.unwrap() {
SyncMessage::Ack { revision } => assert_eq!(revision, rev),
other => panic!("client: expected Ack, got {other:?}"),
}
}
client
.send(&SyncMessage::SyncComplete {
revisions_transferred: total_sent,
bytes_transferred: 0,
})
.await
.unwrap();
let (server_received, dst_final) = server_handle.await.unwrap();
assert_eq!(server_received, total_sent);
(total_sent, dst_final)
}
// ─────────────────────────────────────────────────────────────────────────────
// Tests
// ─────────────────────────────────────────────────────────────────────────────
/// Push 5 revisions to an empty destination over QUIC — all 5 must arrive.
#[tokio::test]
async fn quic_push_all_to_empty() {
let (_src_tmp, src) = make_onion(5);
let (_dst_tmp, dst) = make_onion(0);
let (transferred, dst_final) = run_quic_push(src, dst).await;
assert_eq!(transferred, 5);
assert_eq!(dst_final.revision_count(), 5);
}
/// Push when already in sync — zero packets over QUIC.
#[tokio::test]
async fn quic_push_noop_when_in_sync() {
let (_src_tmp, src) = make_onion(3);
let (_dst_tmp, dst) = make_onion(3);
let (transferred, dst_final) = run_quic_push(src, dst).await;
assert_eq!(transferred, 0);
assert_eq!(dst_final.revision_count(), 3);
}
/// Partial delta over QUIC: N=8, K=3 → 5 packets.
#[tokio::test]
async fn quic_push_partial_delta() {
const K: u8 = 3;
const N: u8 = 8;
let (_src_tmp, src) = make_onion(N);
let (_dst_tmp, dst) = make_onion(K);
let (transferred, dst_final) = run_quic_push(src, dst).await;
assert_eq!(transferred, (N - K) as u64);
assert_eq!(dst_final.revision_count(), N as u64);
}
/// BLAKE3 hashes must survive the QUIC wire intact.
#[tokio::test]
async fn quic_revision_hashes_match_after_sync() {
let (_src_tmp, src) = make_onion(4);
let (_dst_tmp, dst) = make_onion(0);
let src_revisions = src.list_revisions();
let (_, dst_final) = run_quic_push(src, dst).await;
let dst_revisions = dst_final.list_revisions();
assert_eq!(src_revisions.len(), dst_revisions.len());
for (s, d) in src_revisions.iter().zip(dst_revisions.iter()) {
assert_eq!(s.blake3_hex, d.blake3_hex, "rev {} hash mismatch", s.revision);
assert_eq!(s.annotation, d.annotation);
}
}
/// Large push over QUIC: 20→7 = 13 packets.
#[tokio::test]
async fn quic_large_push_correct_count() {
const K: u8 = 7;
const N: u8 = 20;
let (_src_tmp, src) = make_onion(N);
let (_dst_tmp, dst) = make_onion(K);
let (transferred, dst_final) = run_quic_push(src, dst).await;
assert_eq!(transferred, (N - K) as u64);
assert_eq!(dst_final.revision_count(), N as u64);
}
// ─────────────────────────────────────────────────────────────────────────────
// Pull helper: server has `src`; client starts with `dst` and pulls from it.
// Returns (packets_transferred, final_client_onion).
// ─────────────────────────────────────────────────────────────────────────────
async fn run_quic_pull(src: OnionFile, mut dst: OnionFile) -> (u64, OnionFile) {
let (srv_cfg, cli_cfg) = make_quic_configs();
let server = QuicServer::bind(localhost_any(), srv_cfg).await.unwrap();
let server_addr = server.local_addr;
// ── Server task ──────────────────────────────────────────────────────────
let server_handle = tokio::spawn(async move {
let conn = server.accept().await.unwrap();
let client_rev_count = match conn.recv().await.unwrap() {
SyncMessage::ManifestRequest { revision_count, .. } => revision_count,
other => panic!("server: expected ManifestRequest, got {other:?}"),
};
let srv_manifest = ClawSyncManifest::from_onion("srv", &src, H5_BASE);
conn.send(&SyncMessage::ManifestResponse { manifest: srv_manifest })
.await
.unwrap();
if src.revision_count() > client_rev_count {
let remote_head = if client_rev_count == 0 {
NO_PARENT
} else {
client_rev_count - 1
};
let packets = diff_revisions(&src, remote_head).unwrap();
let total = packets.len() as u64;
let mut bytes_sent = 0u64;
for packet in packets {
bytes_sent += packet.page_data_size() as u64;
conn.send(&SyncMessage::LayerPacket { packet }).await.unwrap();
match conn.recv().await.unwrap() {
SyncMessage::Ack { .. } => {}
other => panic!("server: expected Ack, got {other:?}"),
}
}
conn.send(&SyncMessage::SyncComplete {
revisions_transferred: total,
bytes_transferred: bytes_sent,
})
.await
.unwrap();
} else {
match conn.recv().await.unwrap() {
SyncMessage::SyncComplete { .. } => {}
other => panic!("server: expected SyncComplete, got {other:?}"),
}
}
// Quinn resets pending streams when a Connection is dropped. Give the
// client a moment to accept the final SyncComplete stream before this
// task (and thus `conn`) is dropped. This mirrors the pattern used in
// the quic_ack_roundtrip unit test.
tokio::time::sleep(tokio::time::Duration::from_millis(50)).await;
});
// ── Client logic ─────────────────────────────────────────────────────────
let client = quic_connect(server_addr, "localhost", cli_cfg).await.unwrap();
let local_rev_count = dst.revision_count();
client
.send(&SyncMessage::ManifestRequest {
agent_id: "client".to_string(),
head_revision: local_rev_count.saturating_sub(1),
revision_count: local_rev_count,
})
.await
.unwrap();
let server_rev_count = match client.recv().await.unwrap() {
SyncMessage::ManifestResponse { manifest } => manifest.revision_count,
other => panic!("client: expected ManifestResponse, got {other:?}"),
};
if server_rev_count <= local_rev_count {
client
.send(&SyncMessage::SyncComplete {
revisions_transferred: 0,
bytes_transferred: 0,
})
.await
.unwrap();
server_handle.await.unwrap();
return (0, dst);
}
let mut packets = Vec::new();
loop {
match client.recv().await.unwrap() {
SyncMessage::LayerPacket { packet } => {
let rev = packet.revision;
client.send(&SyncMessage::Ack { revision: rev }).await.unwrap();
packets.push(packet);
}
SyncMessage::SyncComplete { revisions_transferred, .. } => {
merge_packets(&mut dst, packets, true).unwrap();
server_handle.await.unwrap();
return (revisions_transferred, dst);
}
other => panic!("client: unexpected {other:?}"),
}
}
}
// ─────────────────────────────────────────────────────────────────────────────
// Pull tests
// ─────────────────────────────────────────────────────────────────────────────
/// Pull all revisions from a full QUIC server to an empty client.
#[tokio::test]
async fn quic_pull_all_from_server() {
let (_srv_tmp, srv) = make_onion(5);
let (_cli_tmp, cli) = make_onion(0);
let (transferred, cli_final) = run_quic_pull(srv, cli).await;
assert_eq!(transferred, 5);
assert_eq!(cli_final.revision_count(), 5);
}
/// Pull when already in sync over QUIC — zero packets.
#[tokio::test]
async fn quic_pull_noop_when_already_in_sync() {
let (_srv_tmp, srv) = make_onion(4);
let (_cli_tmp, cli) = make_onion(4);
let (transferred, cli_final) = run_quic_pull(srv, cli).await;
assert_eq!(transferred, 0);
assert_eq!(cli_final.revision_count(), 4);
}
/// Partial pull over QUIC: server has N, client has K → N-K packets.
#[tokio::test]
async fn quic_pull_partial_delta() {
const K: u8 = 2;
const N: u8 = 7;
let (_srv_tmp, srv) = make_onion(N);
let (_cli_tmp, cli) = make_onion(K);
let (transferred, cli_final) = run_quic_pull(srv, cli).await;
assert_eq!(transferred, (N - K) as u64);
assert_eq!(cli_final.revision_count(), N as u64);
}
/// BLAKE3 hashes must survive the QUIC pull wire path intact.
#[tokio::test]
async fn quic_pull_revision_hashes_match() {
let (_srv_tmp, srv) = make_onion(4);
let (_cli_tmp, cli) = make_onion(0);
let srv_revisions = srv.list_revisions();
let (_, cli_final) = run_quic_pull(srv, cli).await;
let cli_revisions = cli_final.list_revisions();
assert_eq!(srv_revisions.len(), cli_revisions.len());
for (s, c) in srv_revisions.iter().zip(cli_revisions.iter()) {
assert_eq!(s.blake3_hex, c.blake3_hex, "rev {} hash mismatch", s.revision);
assert_eq!(s.annotation, c.annotation);
}
}
+455
View File
@@ -0,0 +1,455 @@
//! End-to-end TCP sync integration test.
//!
//! Scenario: a source node has N revisions; a destination node already has K
//! revisions. The test runs the full wire protocol over a real TCP loopback
//! socket and asserts that exactly N-K layer packets are transferred.
//!
//! Protocol flow (client = source, server = destination):
//! ```text
//! Client Server
//! │── ManifestRequest ─────────▶│
//! │◀─ ManifestResponse ─────────│
//! │── LayerPacket (rev K) ────▶ │ (for each missing revision)
//! │◀─ Ack { revision: K } ──────│
//! │ … │
//! │── SyncComplete ─────────────▶│
//! ```
use std::net::{IpAddr, Ipv4Addr, SocketAddr};
use clawhdf5_onion::writer::OnionFile;
use clawsync_onion::{
differ::diff_revisions,
manifest::ClawSyncManifest,
merger::merge_packets,
};
use clawsync_transport::{
protocol::SyncMessage,
tcp::{TcpConnection, TcpServer},
};
use tempfile::NamedTempFile;
fn localhost_any() -> SocketAddr {
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 0)
}
const H5_BASE: &[u8] = b"\x89HDF\r\n\x1a\n";
/// Create an `OnionFile` with `n` committed revisions.
/// Each revision writes a single page (4 KiB) at offset 0, filled with `i`.
fn make_onion(n: u8) -> (NamedTempFile, OnionFile) {
let tmp = NamedTempFile::new().unwrap();
let h5_path = tmp.path().with_extension("h5");
std::fs::write(&h5_path, H5_BASE).unwrap();
let mut onion = OnionFile::create(&h5_path, 4096).unwrap();
for i in 0..n {
let mut s = onion.begin_session(None).unwrap();
s.record_page(0, &vec![i; 4096]);
onion
.commit_session(s, Some(&format!("rev {i}")))
.unwrap();
}
(tmp, onion)
}
// ─────────────────────────────────────────────────────────────────────────────
// Core helper: run a full push from `src` → `dst` over TCP and return the
// number of LayerPackets actually transferred.
// ─────────────────────────────────────────────────────────────────────────────
async fn run_push(
src: OnionFile,
mut dst: OnionFile,
) -> (u64, OnionFile) {
// Bind the server; the OS picks a free port.
let server = TcpServer::bind(localhost_any()).await.unwrap();
let server_addr = server.local_addr;
// Build the client-side manifest now (before moving `src` into the task).
let src_manifest = ClawSyncManifest::from_onion("src", &src, H5_BASE);
// ── Server task ──────────────────────────────────────────────────────────
// Returns (packets_received, dst) so the caller can inspect the result.
let server_handle = tokio::spawn(async move {
let (mut conn, _) = server.accept().await.unwrap();
let mut packets_received: u64 = 0;
loop {
let msg = conn.recv().await.unwrap();
match msg {
SyncMessage::ManifestRequest { head_revision, .. } => {
// Build our local manifest and send it back.
let dst_manifest =
ClawSyncManifest::from_onion("dst", &dst, H5_BASE);
conn.send(&SyncMessage::ManifestResponse {
manifest: dst_manifest,
})
.await
.unwrap();
let _ = head_revision; // used for negotiation by client
}
SyncMessage::LayerPacket { packet } => {
let rev = packet.revision;
merge_packets(&mut dst, vec![packet], false).unwrap();
packets_received += 1;
conn.send(&SyncMessage::Ack { revision: rev })
.await
.unwrap();
}
SyncMessage::SyncComplete { .. } => {
// Flush and stop accepting.
dst.flush().unwrap();
break;
}
other => panic!("server: unexpected message: {other:?}"),
}
}
(packets_received, dst)
});
// ── Client logic ─────────────────────────────────────────────────────────
let mut client = TcpConnection::connect(server_addr).await.unwrap();
// 1. Send manifest request.
client
.send(&SyncMessage::ManifestRequest {
agent_id: "src".to_string(),
head_revision: src_manifest.head_revision,
revision_count: src_manifest.revision_count,
})
.await
.unwrap();
// 2. Receive remote manifest; determine what to push.
let remote_head = match client.recv().await.unwrap() {
SyncMessage::ManifestResponse { manifest } => {
if manifest.revision_count == 0 {
clawhdf5_onion::format::NO_PARENT
} else {
manifest.head_revision
}
}
other => panic!("client: expected ManifestResponse, got {other:?}"),
};
// 3. Compute and send missing revisions.
let packets = diff_revisions(&src, remote_head).unwrap();
let total_sent = packets.len() as u64;
for packet in packets {
let rev = packet.revision;
client
.send(&SyncMessage::LayerPacket { packet })
.await
.unwrap();
// Wait for ack before sending next packet (ordered, reliable).
match client.recv().await.unwrap() {
SyncMessage::Ack { revision } => assert_eq!(revision, rev),
other => panic!("client: expected Ack, got {other:?}"),
}
}
// 4. Signal completion.
client
.send(&SyncMessage::SyncComplete {
revisions_transferred: total_sent,
bytes_transferred: 0,
})
.await
.unwrap();
let (server_received, dst_final) = server_handle.await.unwrap();
assert_eq!(server_received, total_sent, "server count must match client send count");
(total_sent, dst_final)
}
// ─────────────────────────────────────────────────────────────────────────────
// Tests
// ─────────────────────────────────────────────────────────────────────────────
/// Push 5 revisions to an empty destination — all 5 must be transferred.
#[tokio::test]
async fn push_all_revisions_to_empty_destination() {
let (_src_tmp, src) = make_onion(5);
let (_dst_tmp, dst) = make_onion(0);
let (transferred, dst_final) = run_push(src, dst).await;
assert_eq!(transferred, 5, "all 5 revisions should be transferred");
assert_eq!(dst_final.revision_count(), 5);
}
/// Push when destination is already at the same HEAD — nothing to transfer.
#[tokio::test]
async fn push_when_already_in_sync() {
let (_src_tmp, src) = make_onion(3);
let (_dst_tmp, dst) = make_onion(3);
let (transferred, dst_final) = run_push(src, dst).await;
assert_eq!(transferred, 0, "nothing should be transferred when in sync");
assert_eq!(dst_final.revision_count(), 3);
}
/// Destination has K revisions, source has N; exactly N-K should transfer.
#[tokio::test]
async fn push_partial_delta_transfers_exactly_n_minus_k() {
const K: u8 = 3;
const N: u8 = 8;
let (_src_tmp, src) = make_onion(N);
let (_dst_tmp, dst) = make_onion(K);
let (transferred, dst_final) = run_push(src, dst).await;
let expected = (N - K) as u64;
assert_eq!(
transferred, expected,
"exactly N-K={expected} revisions should be transferred"
);
assert_eq!(dst_final.revision_count(), N as u64);
}
/// After sync the destination can reconstruct every revision that the source has.
#[tokio::test]
async fn destination_revisions_match_source_after_sync() {
let (_src_tmp, src) = make_onion(4);
let (_dst_tmp, dst) = make_onion(0);
// Capture source revision metadata before the move.
let src_revisions = src.list_revisions();
let (_transferred, dst_final) = run_push(src, dst).await;
let dst_revisions = dst_final.list_revisions();
assert_eq!(src_revisions.len(), dst_revisions.len());
for (s, d) in src_revisions.iter().zip(dst_revisions.iter()) {
assert_eq!(s.revision, d.revision, "revision numbers must match");
assert_eq!(
s.annotation, d.annotation,
"annotations must survive the wire"
);
assert_eq!(
s.blake3_hex, d.blake3_hex,
"BLAKE3 hashes must be identical — page data is intact"
);
}
}
/// Verify idempotency: pushing the same revisions twice leaves the destination
/// with exactly the same number of revisions (no duplicates).
#[tokio::test]
async fn push_twice_is_idempotent() {
let (_src_tmp1, src1) = make_onion(3);
let (_src_tmp2, _src2) = make_onion(3);
let (_dst_tmp, dst) = make_onion(0);
// First push.
let (_t1, dst_after_first) = run_push(src1, dst).await;
assert_eq!(dst_after_first.revision_count(), 3);
// Second push of the identical 3 revisions.
// `run_push` moves `dst_after_first`, so we need to rebuild one.
// Re-open the same .onion file from disk.
let (_src_tmp3, src3) = make_onion(3);
let (_dst_tmp2, dst2) = make_onion(3); // simulates "already has all"
let (t2, dst_final) = run_push(src3, dst2).await;
assert_eq!(t2, 0, "second push should transfer nothing");
assert_eq!(dst_final.revision_count(), 3, "no duplicate revisions");
let _ = dst_after_first; // keep alive until here
}
/// Large push: 20 revisions, destination has 7 — assert 13 packets.
#[tokio::test]
async fn large_push_correct_packet_count() {
const K: u8 = 7;
const N: u8 = 20;
let (_src_tmp, src) = make_onion(N);
let (_dst_tmp, dst) = make_onion(K);
let (transferred, dst_final) = run_push(src, dst).await;
assert_eq!(transferred, (N - K) as u64);
assert_eq!(dst_final.revision_count(), N as u64);
}
// ─────────────────────────────────────────────────────────────────────────────
// Pull helper: server has `src`; client starts with `dst` and pulls from it.
// Returns (packets_transferred, final_client_onion).
// ─────────────────────────────────────────────────────────────────────────────
async fn run_pull(
src: OnionFile, // server-side onion (richer)
mut dst: OnionFile, // client-side onion (missing revisions)
) -> (u64, OnionFile) {
let server = TcpServer::bind(localhost_any()).await.unwrap();
let server_addr = server.local_addr;
// ── Server task ──────────────────────────────────────────────────────────
let server_handle = tokio::spawn(async move {
let (mut conn, _) = server.accept().await.unwrap();
// 1. Receive client's ManifestRequest.
let client_rev_count = match conn.recv().await.unwrap() {
SyncMessage::ManifestRequest { revision_count, .. } => revision_count,
other => panic!("server: expected ManifestRequest, got {other:?}"),
};
// 2. Send back the server manifest.
let srv_manifest = ClawSyncManifest::from_onion("srv", &src, H5_BASE);
conn.send(&SyncMessage::ManifestResponse { manifest: srv_manifest })
.await
.unwrap();
// 3. Server has more revisions — push the delta to the client.
if src.revision_count() > client_rev_count {
let remote_head = if client_rev_count == 0 {
clawhdf5_onion::format::NO_PARENT
} else {
client_rev_count - 1
};
let packets = diff_revisions(&src, remote_head).unwrap();
let total = packets.len() as u64;
let mut bytes_sent = 0u64;
for packet in packets {
bytes_sent += packet.page_data_size() as u64;
conn.send(&SyncMessage::LayerPacket { packet }).await.unwrap();
match conn.recv().await.unwrap() {
SyncMessage::Ack { .. } => {}
other => panic!("server: expected Ack, got {other:?}"),
}
}
conn.send(&SyncMessage::SyncComplete {
revisions_transferred: total,
bytes_transferred: bytes_sent,
})
.await
.unwrap();
} else {
// Already in sync: wait for client's SyncComplete.
match conn.recv().await.unwrap() {
SyncMessage::SyncComplete { .. } => {}
other => panic!("server: expected SyncComplete, got {other:?}"),
}
}
});
// ── Client logic ─────────────────────────────────────────────────────────
let mut client = TcpConnection::connect(server_addr).await.unwrap();
let local_rev_count = dst.revision_count();
// 1. Send manifest request.
client
.send(&SyncMessage::ManifestRequest {
agent_id: "client".to_string(),
head_revision: local_rev_count.saturating_sub(1),
revision_count: local_rev_count,
})
.await
.unwrap();
// 2. Receive server manifest; check if there is anything to pull.
let server_rev_count = match client.recv().await.unwrap() {
SyncMessage::ManifestResponse { manifest } => manifest.revision_count,
other => panic!("client: expected ManifestResponse, got {other:?}"),
};
if server_rev_count <= local_rev_count {
// Nothing to pull — tell server we're done.
client
.send(&SyncMessage::SyncComplete {
revisions_transferred: 0,
bytes_transferred: 0,
})
.await
.unwrap();
server_handle.await.unwrap();
return (0, dst);
}
// 3. Receive packets from server.
let mut packets = Vec::new();
loop {
match client.recv().await.unwrap() {
SyncMessage::LayerPacket { packet } => {
let rev = packet.revision;
client.send(&SyncMessage::Ack { revision: rev }).await.unwrap();
packets.push(packet);
}
SyncMessage::SyncComplete { revisions_transferred, .. } => {
merge_packets(&mut dst, packets, true).unwrap();
server_handle.await.unwrap();
return (revisions_transferred, dst);
}
other => panic!("client: unexpected {other:?}"),
}
}
}
// ─────────────────────────────────────────────────────────────────────────────
// Pull tests
// ─────────────────────────────────────────────────────────────────────────────
/// Pull all revisions from a full server to an empty client.
#[tokio::test]
async fn pull_all_from_server() {
let (_srv_tmp, srv) = make_onion(5);
let (_cli_tmp, cli) = make_onion(0);
let (transferred, cli_final) = run_pull(srv, cli).await;
assert_eq!(transferred, 5);
assert_eq!(cli_final.revision_count(), 5);
}
/// Pull when client and server are already in sync — zero packets.
#[tokio::test]
async fn pull_noop_when_already_in_sync() {
let (_srv_tmp, srv) = make_onion(4);
let (_cli_tmp, cli) = make_onion(4);
let (transferred, cli_final) = run_pull(srv, cli).await;
assert_eq!(transferred, 0, "nothing to pull when already in sync");
assert_eq!(cli_final.revision_count(), 4);
}
/// Pull partial delta: server has N, client has K — client receives N-K packets.
#[tokio::test]
async fn pull_partial_delta() {
const K: u8 = 2;
const N: u8 = 7;
let (_srv_tmp, srv) = make_onion(N);
let (_cli_tmp, cli) = make_onion(K);
let (transferred, cli_final) = run_pull(srv, cli).await;
assert_eq!(transferred, (N - K) as u64);
assert_eq!(cli_final.revision_count(), N as u64);
}
/// BLAKE3 hashes must survive the pull wire path intact.
#[tokio::test]
async fn pull_revision_hashes_match() {
let (_srv_tmp, srv) = make_onion(4);
let (_cli_tmp, cli) = make_onion(0);
let srv_revisions = srv.list_revisions();
let (_, cli_final) = run_pull(srv, cli).await;
let cli_revisions = cli_final.list_revisions();
assert_eq!(srv_revisions.len(), cli_revisions.len());
for (s, c) in srv_revisions.iter().zip(cli_revisions.iter()) {
assert_eq!(s.blake3_hex, c.blake3_hex, "rev {} hash mismatch", s.revision);
assert_eq!(s.annotation, c.annotation);
}
}