docs: changelog and CLAUDE.md for the durability & integrity work
Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
bf8bbec87e
commit
005f37e846
@@ -19,6 +19,54 @@
|
||||
- `clawhdf5-agent`: `benches/bench.rs` and `benches/memory_bench.rs` no longer
|
||||
compiled against the current `strategy`/`consolidation` APIs.
|
||||
|
||||
### Durability & Integrity
|
||||
- `clawhdf5-agent`: a crash between writing a checkpoint and truncating the WAL
|
||||
no longer **duplicates every pending entry** on the next open. Each
|
||||
checkpoint records a `WalMark` (byte length + chained CRC of the WAL prefix it
|
||||
folded in) in `/meta`; `open()` skips exactly that prefix when it is still
|
||||
present. No WAL format change for this; older files behave as before.
|
||||
- `clawhdf5-agent`: checkpoints and snapshots are durable as a unit — the temp
|
||||
file is synced before the rename and the directory after it. Individual WAL
|
||||
appends remain unsynced by design (documented in `CLAUDE.md`).
|
||||
- `clawhdf5-agent`: `save_or_update` hits are logged as a new `Update` WAL
|
||||
record, so replay updates in place instead of appending a duplicate. WAL
|
||||
header version 3 → 4 (so older builds refuse the file rather than truncating
|
||||
a record they can't parse); v3 files are read and upgraded in place.
|
||||
- `clawhdf5-agent`: loading validates every per-record dataset length (a
|
||||
truncated store is now `MemoryError::Schema`, not a later panic), fixes the
|
||||
`n.len() == n.len()` tautology that trusted a norms dataset of any length,
|
||||
and rejects `embedding_dim == 0` with records present.
|
||||
- `clawhdf5-agent`: eight behavioural `MemoryConfig` fields are now persisted in
|
||||
`/meta`. Previously they reset to defaults on every open — a compressed store
|
||||
was rewritten uncompressed, `wal_enabled = false` flipped back to `true`.
|
||||
- `clawhdf5-agent`: `compression = true` never worked in a default build (it
|
||||
requested Zstd without enabling the feature, so every checkpoint failed with
|
||||
`unsupported filter: 32015`). Default builds now use deflate; Zstd is the new
|
||||
opt-in `zstd` feature.
|
||||
- `clawhdf5-agent`: **single-writer lock** (`<store>.h5.lock`,
|
||||
`MemoryError::Locked`) — two handles on one store used to silently destroy
|
||||
each other's data. New `HDF5Memory::open_read_only` gives a lock-free,
|
||||
never-writing view; the CLI's read-only subcommands use it.
|
||||
- `clawhdf5-agent`: an unreadable WAL (torn header / bad magic) is quarantined
|
||||
(`HDF5Memory::quarantined_wal()`) instead of blocking `open()` of a healthy
|
||||
store. A WAL from an unknown newer version still fails and is left intact.
|
||||
- `clawhdf5-agent`: provenance records are renumbered on compaction (they
|
||||
weren't, so every later `save_or_update` raised a false High integrity
|
||||
alert); pending anomaly alerts and tracked sessions are bounded;
|
||||
`snapshot()` includes entries still in the WAL.
|
||||
- `clawhdf5-agent`: hybrid ranking is deterministic (index tie-breaks instead
|
||||
of `HashMap` order); a set of identical positive scores — including a single
|
||||
candidate — normalises to 1.0 rather than 0.0; the Hebbian boost no longer
|
||||
reinforces zero-score filler results.
|
||||
- `clawhdf5-format`: chunked/VDS/hyperslab reads size their buffers with
|
||||
overflow-checked arithmetic and fallible allocation, so crafted dimensions
|
||||
are `FormatError::Overflow` instead of a wrapped size or a process abort;
|
||||
`parallel_read` bounds checks use `checked_add`.
|
||||
- `clawhdf5`: a malformed filter-pipeline message is an error instead of being
|
||||
treated as "no filters" (which returned compressed bytes as data);
|
||||
`FileBuilder::write` is atomic and synced instead of truncating the
|
||||
destination first.
|
||||
|
||||
### CI / Testing
|
||||
- CI now lints every target (`cargo clippy --all-targets`) plus
|
||||
`clawhdf5-format`'s optional features, compiles all benches, and tests the
|
||||
|
||||
Reference in New Issue
Block a user