feat(agent): MemoryConfig::float16 stores half-precision embeddings
CI / test-arm64 (pull_request) Successful in 1m19s
CI / test (pull_request) Successful in 4m58s

The setting was persisted in /meta and otherwise ignored: embeddings
were always written as f32. It now does what it says.

clawhdf5-format:
- `DatasetBuilder::with_f16_data` writes IEEE binary16 (numpy float16),
  rounding to nearest-even, and `make_f16_type`.
- `clawhdf5_format::float16` holds the f32 <-> f16 conversions, the one
  implementation the writer, the reader and the agent all use. Checked
  against the `half` crate on 16.7M f32 values and round-trips all 65536
  half values; the h5py interop tests confirm the rounding matches
  numpy's bit for bit (4020 values incl. ties, subnormals, overflow).
- Reading little-endian float16 as f32 has a fast path.

clawhdf5-agent:
- A float16 store writes /memory/embeddings as half precision, and
  `MemoryCache::half_precision` rounds each embedding as it enters the
  cache (save, update, WAL replay, and on load of a store still f32 on
  disk), so memory and file agree bit for bit and a store searches the
  same before and after a reopen (tested).
- Values beyond +-65504 are refused with the new
  `MemoryError::InvalidEntry` rather than stored as infinity, on every
  save path; batches are all or nothing, and a rejected ephemeral entry
  stays in the ephemeral tier. Breaking for exhaustive matches.
- CLI: `create --float16`. Off by default.

Measured on tank, 384-dim, six runs alternating order, medians
(search_harness --float16-study --full): at 100K the file goes from
154.0 to 80.8 MiB (-48%), checkpoint 752 -> 512 ms, open 300 -> 252 ms;
vector recall@10 against an exact scan and hybrid_search latency do not
change. At 10K open is 3 ms slower. Also a test that h5py opens a whole
agent store, f32 and float16, and decodes every dataset.

Docs: README, BENCHMARKS.md ("float16 embedding storage"), CHANGELOG
(including the h5py interop fixes in the previous commit), CLAUDE.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-24 12:00:38 -05:00
co-authored by Claude Opus 5.5
parent 5e4aa1c6bf
commit d0db83812b
18 changed files with 1111 additions and 43 deletions
+51
View File
@@ -3,6 +3,23 @@
## Unreleased
### Upgrade Notes
- **Files written by clawhdf5 now open in h5py and libhdf5.** Every `f32`
dataset we wrote — including every agent store's embeddings — was refused
with "sign bit position out of bounds", and every empty dataset with
"invalid dataset size". Both were write-side bugs present in every release;
clawhdf5's own reader was unaffected. An agent store is rewritten in full at
each checkpoint, so it becomes readable at its next checkpoint on this
version; other files with `f32` or empty datasets need rewriting. Details in
`docs/known-issues.md`.
- **`MemoryConfig::float16` now does what it says.** It was persisted and
otherwise ignored; embeddings were always stored as `f32`. A store created
with it on now writes half-precision embeddings (48% smaller files) and
rounds embeddings to half precision as they are saved. A store that already
had `float16 = true` rounds its embeddings when next opened and writes them
as `float16` at its next checkpoint. Off by default.
- **Breaking:** `MemoryError` gained `InvalidEntry`, returned when a
`float16` store is given an embedding value beyond ±65504. Exhaustive
matches need the new arm.
- **The default build no longer compiles any C.** Deflate now defaults to the
pure-Rust zlib-rs instead of zlib-ng, so building the core crates needs
neither cmake nor a C compiler. Speed on HDF5 reads and writes is within 6%
@@ -23,6 +40,40 @@
`quantized_index = false`, or pass `create --f32-index` to the CLI, to opt
out. The CLI's `--quantized-index` is still accepted but is now a no-op.
### Interop
- `clawhdf5-format`: **every `f32` dataset was unreadable by h5py and
libhdf5.** The float datatype encoder hard-coded the sign bit's position to
63, correct only for `f64`; libhdf5 validates it and refused the dataset. It
is now derived from the type (15 / 31 / 63). Our reader ignores the field,
and the interop suites only wrote `f64`, which is how it went unnoticed.
- `clawhdf5-format`: **every empty dataset was unreadable by h5py and
libhdf5.** It was written with a real address and zero bytes, which trips
libhdf5's `addr + size <= addr` overflow check. An empty contiguous dataset
now gets the undefined address, as libhdf5 writes it. This affected every
agent store without sessions or a knowledge graph.
- New interop tests: `f32` and `float16` datasets in both directions (our
`float16` rounding matches numpy's bit for bit on 4 020 probe values,
including ties, subnormals and the overflow boundary), and an agent store —
`f32` and `float16` — opened by h5py with every dataset decoded.
### Storage
- `clawhdf5-format`: **half-precision datasets.**
`DatasetBuilder::with_f16_data` writes IEEE binary16 (numpy `float16`),
rounding to nearest-even; `make_f16_type`, and `clawhdf5_format::float16`
with the conversions, which are checked against the `half` crate on 16.7M
values and round-trip all 65 536 half values. Reading `float16` as `f32`
gained a little-endian fast path.
- `clawhdf5-agent`: **`MemoryConfig::float16` stores embeddings as half
precision.** At 100K x 384 the file goes from 154.0 to 80.8 MiB (−48%), a
checkpoint from 752 to 512 ms and open from 300 to 252 ms, with the same
vector recall@10 against an exact scan (0.999 vs 0.994) and the same
`hybrid_search` latency; at 10K open is 3 ms slower. The cache rounds each
embedding as it is saved, so memory and file agree bit for bit and a store
returns the same results before and after a reopen (tested). Out-of-range
values are refused with `MemoryError::InvalidEntry` rather than stored as
infinity; batches are all or nothing. CLI: `create --float16`. See
`BENCHMARKS.md`, "float16 embedding storage".
### Build
- **Pure-Rust default.** `clawhdf5-format`, `clawhdf5-filters` and the
`clawhdf5` facade default to the `zlib-rs` deflate backend; `fast-deflate`