feat(agent): MemoryConfig::float16 stores half-precision embeddings
CI / test-arm64 (pull_request) Successful in 1m19s
CI / test (pull_request) Successful in 4m58s

The setting was persisted in /meta and otherwise ignored: embeddings
were always written as f32. It now does what it says.

clawhdf5-format:
- `DatasetBuilder::with_f16_data` writes IEEE binary16 (numpy float16),
  rounding to nearest-even, and `make_f16_type`.
- `clawhdf5_format::float16` holds the f32 <-> f16 conversions, the one
  implementation the writer, the reader and the agent all use. Checked
  against the `half` crate on 16.7M f32 values and round-trips all 65536
  half values; the h5py interop tests confirm the rounding matches
  numpy's bit for bit (4020 values incl. ties, subnormals, overflow).
- Reading little-endian float16 as f32 has a fast path.

clawhdf5-agent:
- A float16 store writes /memory/embeddings as half precision, and
  `MemoryCache::half_precision` rounds each embedding as it enters the
  cache (save, update, WAL replay, and on load of a store still f32 on
  disk), so memory and file agree bit for bit and a store searches the
  same before and after a reopen (tested).
- Values beyond +-65504 are refused with the new
  `MemoryError::InvalidEntry` rather than stored as infinity, on every
  save path; batches are all or nothing, and a rejected ephemeral entry
  stays in the ephemeral tier. Breaking for exhaustive matches.
- CLI: `create --float16`. Off by default.

Measured on tank, 384-dim, six runs alternating order, medians
(search_harness --float16-study --full): at 100K the file goes from
154.0 to 80.8 MiB (-48%), checkpoint 752 -> 512 ms, open 300 -> 252 ms;
vector recall@10 against an exact scan and hybrid_search latency do not
change. At 10K open is 3 ms slower. Also a test that h5py opens a whole
agent store, f32 and float16, and decodes every dataset.

Docs: README, BENCHMARKS.md ("float16 embedding storage"), CHANGELOG
(including the h5py interop fixes in the previous commit), CLAUDE.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-24 12:00:38 -05:00
co-authored by Claude Opus 5.5
parent 5e4aa1c6bf
commit d0db83812b
18 changed files with 1111 additions and 43 deletions
+34
View File
@@ -149,6 +149,40 @@ vectors, and recall is measured against brute-force ground truth rather than
against the f32 index, whose own approximation errors a re-scored search is
entitled to get right.
### float16 embedding storage (`MemoryConfig::float16`)
Measured 2026-09-23 on tank (AMD Ryzen 7 7800X3D). The same clustered
384-dim data in an `f32` store and a `float16` store, both with the default
int8 index and Hebbian boosting off (so every query sees the same store).
Recall is vector-only `hybrid_search` against an exact scan of the original
`f32` vectors, 200 queries. Six runs, three with each store going first;
medians. Nothing depended on the order.
```bash
cargo run --release -p clawhdf5-bench --bin search_harness -- --float16-study --full
cargo run --release -p clawhdf5-bench --bin search_harness -- --float16-study --full --f16-first
```
| N | embeddings | file MiB | checkpoint ms | open ms | recall@10 | top-10 overlap | hybrid p50 ms |
|---:|---|---:|---:|---:|---:|---:|---:|
| 1 000 | f32 | 1.6 | 8 | 1.6 | 1.0000 | | 0.072 |
| 1 000 | float16 | 0.8 | 6 | 1.9 | 0.9980 | 0.9980 | 0.072 |
| 10 000 | f32 | 15.4 | 66 | 15.2 | 1.0000 | | 0.495 |
| 10 000 | float16 | 8.2 | 45 | 18.1 | 1.0000 | 1.0000 | 0.494 |
| 100 000 | f32 | 154.0 | 752 | 299.8 | 0.9940 | | 4.676 |
| 100 000 | float16 | **80.8** | **512** | **252.1** | 0.9990 | 0.9940 | 4.654 |
The file is 48% smaller, checkpoints write less and open reads less. At
small N opening is slightly slower (widening halves costs more than the I/O it
saves: +3 ms at 10K). Recall does not move: half precision keeps about three
significant digits, far finer than the gaps between neighbours on unit-length
embeddings. Recall was identical in every run; the 0.999 against 0.994 at
100K is two slightly different HNSW graphs, not an improvement to claim.
The in-memory cache holds the half-rounded values, so the store searches the
same before and after a reopen; RAM use is unchanged (the cache is still
`f32`). What `float16` saves is disk, and the I/O that goes with it.
### Opening a store (`read_from_disk`)
`HDF5Memory::open` memory-mapped the file, copied the whole mapping into a