feat(agent): MemoryConfig::float16 stores half-precision embeddings
The setting was persisted in /meta and otherwise ignored: embeddings
were always written as f32. It now does what it says.
clawhdf5-format:
- `DatasetBuilder::with_f16_data` writes IEEE binary16 (numpy float16),
rounding to nearest-even, and `make_f16_type`.
- `clawhdf5_format::float16` holds the f32 <-> f16 conversions, the one
implementation the writer, the reader and the agent all use. Checked
against the `half` crate on 16.7M f32 values and round-trips all 65536
half values; the h5py interop tests confirm the rounding matches
numpy's bit for bit (4020 values incl. ties, subnormals, overflow).
- Reading little-endian float16 as f32 has a fast path.
clawhdf5-agent:
- A float16 store writes /memory/embeddings as half precision, and
`MemoryCache::half_precision` rounds each embedding as it enters the
cache (save, update, WAL replay, and on load of a store still f32 on
disk), so memory and file agree bit for bit and a store searches the
same before and after a reopen (tested).
- Values beyond +-65504 are refused with the new
`MemoryError::InvalidEntry` rather than stored as infinity, on every
save path; batches are all or nothing, and a rejected ephemeral entry
stays in the ephemeral tier. Breaking for exhaustive matches.
- CLI: `create --float16`. Off by default.
Measured on tank, 384-dim, six runs alternating order, medians
(search_harness --float16-study --full): at 100K the file goes from
154.0 to 80.8 MiB (-48%), checkpoint 752 -> 512 ms, open 300 -> 252 ms;
vector recall@10 against an exact scan and hybrid_search latency do not
change. At 10K open is 3 ms slower. Also a test that h5py opens a whole
agent store, f32 and float16, and decodes every dataset.
Docs: README, BENCHMARKS.md ("float16 embedding storage"), CHANGELOG
(including the h5py interop fixes in the previous commit), CLAUDE.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -3,6 +3,23 @@
|
||||
## Unreleased
|
||||
|
||||
### Upgrade Notes
|
||||
- **Files written by clawhdf5 now open in h5py and libhdf5.** Every `f32`
|
||||
dataset we wrote — including every agent store's embeddings — was refused
|
||||
with "sign bit position out of bounds", and every empty dataset with
|
||||
"invalid dataset size". Both were write-side bugs present in every release;
|
||||
clawhdf5's own reader was unaffected. An agent store is rewritten in full at
|
||||
each checkpoint, so it becomes readable at its next checkpoint on this
|
||||
version; other files with `f32` or empty datasets need rewriting. Details in
|
||||
`docs/known-issues.md`.
|
||||
- **`MemoryConfig::float16` now does what it says.** It was persisted and
|
||||
otherwise ignored; embeddings were always stored as `f32`. A store created
|
||||
with it on now writes half-precision embeddings (48% smaller files) and
|
||||
rounds embeddings to half precision as they are saved. A store that already
|
||||
had `float16 = true` rounds its embeddings when next opened and writes them
|
||||
as `float16` at its next checkpoint. Off by default.
|
||||
- **Breaking:** `MemoryError` gained `InvalidEntry`, returned when a
|
||||
`float16` store is given an embedding value beyond ±65504. Exhaustive
|
||||
matches need the new arm.
|
||||
- **The default build no longer compiles any C.** Deflate now defaults to the
|
||||
pure-Rust zlib-rs instead of zlib-ng, so building the core crates needs
|
||||
neither cmake nor a C compiler. Speed on HDF5 reads and writes is within 6%
|
||||
@@ -23,6 +40,40 @@
|
||||
`quantized_index = false`, or pass `create --f32-index` to the CLI, to opt
|
||||
out. The CLI's `--quantized-index` is still accepted but is now a no-op.
|
||||
|
||||
### Interop
|
||||
- `clawhdf5-format`: **every `f32` dataset was unreadable by h5py and
|
||||
libhdf5.** The float datatype encoder hard-coded the sign bit's position to
|
||||
63, correct only for `f64`; libhdf5 validates it and refused the dataset. It
|
||||
is now derived from the type (15 / 31 / 63). Our reader ignores the field,
|
||||
and the interop suites only wrote `f64`, which is how it went unnoticed.
|
||||
- `clawhdf5-format`: **every empty dataset was unreadable by h5py and
|
||||
libhdf5.** It was written with a real address and zero bytes, which trips
|
||||
libhdf5's `addr + size <= addr` overflow check. An empty contiguous dataset
|
||||
now gets the undefined address, as libhdf5 writes it. This affected every
|
||||
agent store without sessions or a knowledge graph.
|
||||
- New interop tests: `f32` and `float16` datasets in both directions (our
|
||||
`float16` rounding matches numpy's bit for bit on 4 020 probe values,
|
||||
including ties, subnormals and the overflow boundary), and an agent store —
|
||||
`f32` and `float16` — opened by h5py with every dataset decoded.
|
||||
|
||||
### Storage
|
||||
- `clawhdf5-format`: **half-precision datasets.**
|
||||
`DatasetBuilder::with_f16_data` writes IEEE binary16 (numpy `float16`),
|
||||
rounding to nearest-even; `make_f16_type`, and `clawhdf5_format::float16`
|
||||
with the conversions, which are checked against the `half` crate on 16.7M
|
||||
values and round-trip all 65 536 half values. Reading `float16` as `f32`
|
||||
gained a little-endian fast path.
|
||||
- `clawhdf5-agent`: **`MemoryConfig::float16` stores embeddings as half
|
||||
precision.** At 100K x 384 the file goes from 154.0 to 80.8 MiB (−48%), a
|
||||
checkpoint from 752 to 512 ms and open from 300 to 252 ms, with the same
|
||||
vector recall@10 against an exact scan (0.999 vs 0.994) and the same
|
||||
`hybrid_search` latency; at 10K open is 3 ms slower. The cache rounds each
|
||||
embedding as it is saved, so memory and file agree bit for bit and a store
|
||||
returns the same results before and after a reopen (tested). Out-of-range
|
||||
values are refused with `MemoryError::InvalidEntry` rather than stored as
|
||||
infinity; batches are all or nothing. CLI: `create --float16`. See
|
||||
`BENCHMARKS.md`, "float16 embedding storage".
|
||||
|
||||
### Build
|
||||
- **Pure-Rust default.** `clawhdf5-format`, `clawhdf5-filters` and the
|
||||
`clawhdf5` facade default to the `zlib-rs` deflate backend; `fast-deflate`
|
||||
|
||||
Reference in New Issue
Block a user