Phase 4d: WAL segment rotation

Browse Source

Turns the single-file Phase 4c WAL into a segmented log so it
can grow past a single file safely. This unblocks every
downstream Phase 4d/4e integration — reconnect + push loop
can't rely on an unbounded single file.

Layout change:
  <root>/segment-<20-digit-first-seq>.bin

20-digit zero-padded first-seq means lex sort == numeric sort,
so `read_dir + sort_by_key` recovers the natural order.

Rotation policy:
  * `max_segment_bytes` default 8 MiB, overridable via
    `open_with_options`.
  * `append` rolls to a fresh segment BEFORE writing when the
    current tail is non-empty AND at/above the cap. A single
    oversize record always lands in one segment — we never split
    a record.

Truncation across segments:
  * whole segments with `last_seq <= watermark` are `unlink`'d
  * the boundary segment (if any) is rewritten in place via
    `tempfile-in-parent + rename` + parent-dir fsync
  * full truncation resets head/tail to 0 and the next append
    creates a fresh segment

Legacy compat: on open, if a pre-4d `log.bin` is present and
no `segment-*.bin` files exist, it is scanned for its first
seq and renamed to the correct segment name. Refuses to
silently overwrite on filename collision.

Tests (18, all green): rotation-happens-at-cap, reopen-
enumerates-all-segments, truncate-drops-whole-segments,
truncate-partial-rewrites-boundary, oversize-record-still-
fits-one-segment, legacy-log.bin-migration, plus the full
Phase 4c suite (fresh open, append, iter partial ranges,
reopen recovers tail, torn-write truncation, corruption is
hard error, full truncation appendable, below-head no-op,
large payload, empty payload, append-after-reopen).

942-line file, comfortably under the 1300-line ceiling.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

...

This commit is contained in:

Omar Sobh

2026-07-14 00:47:19 -07:00

co-authored by Claude Opus 4.7

parent dd1a37fe7e

commit 0cf00e5954

1 changed files with 568 additions and 227 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

claw-store/src/cluster/wal.rs

+568 -227

File diff suppressed because it is too large Load Diff