feat(agent): new stores default to float16 embeddings

MemoryConfig::float16 now defaults to true for new stores, on
measurement: on the full LongMemEval haystack with real MiniLM
embeddings every retrieval metric matched f32 (previous commit), and at
100K the file is 48% smaller with faster checkpoints and opens.

Existing stores are unaffected: every agent store has recorded
`float16 = false` in /meta and keeps it. A test opens the v2.5.0
fixture, saves and checkpoints, and checks the embeddings are still f32
with the old rows bit-identical; another checks a new store is float16.

CLI: `create --f32` opts out; like `--f32-index` it only ever switches
the default off. `--float16` is still accepted and now a no-op.
Values beyond +-65504 are refused, so f32 remains the choice for
unnormalised vectors — the upgrade note says so.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-24 19:39:33 -05:00
co-authored by Claude Opus 5.5
parent dbaf3f505d
commit 5c8323cb1e
6 changed files with 109 additions and 25 deletions
+17 -6
View File
@@ -11,12 +11,18 @@
each checkpoint, so it becomes readable at its next checkpoint on this
version; other files with `f32` or empty datasets need rewriting. Details in
`docs/known-issues.md`.
- **`MemoryConfig::float16` now does what it says.** It was persisted and
otherwise ignored; embeddings were always stored as `f32`. A store created
with it on now writes half-precision embeddings (48% smaller files) and
rounds embeddings to half precision as they are saved. A store that already
had `float16 = true` rounds its embeddings when next opened and writes them
as `float16` at its next checkpoint. Off by default.
- **New stores store embeddings as half precision by default.**
`MemoryConfig::float16` was persisted and otherwise ignored; it now writes
`float16` embeddings (48% smaller files at 100K) and rounds each embedding
to half precision as it is saved — and it defaults to `true` for new
stores. On the full LongMemEval haystack with real MiniLM embeddings every
retrieval metric matched `f32`. **Existing stores are unaffected**: every
agent store has recorded `float16 = false`, and keeps it (a v2.5.0 fixture
guards this). A store that already had `float16 = true` rounds its
embeddings when next opened and writes them as `float16` at its next
checkpoint. Opt out with `float16 = false` or `create --f32`; the CLI's
`--float16` is still accepted and now a no-op. Values beyond ±65504 are
refused, so keep `f32` for unnormalised vectors.
- **Breaking:** `MemoryError` gained `InvalidEntry`, returned when a
`float16` store is given an embedding value beyond ±65504. Exhaustive
matches need the new arm.
@@ -123,6 +129,11 @@
now an error.
### Defaults
- `clawhdf5-agent`: `MemoryConfig::float16` defaults to `true` for new stores,
measured rather than assumed: identical LongMemEval retrieval on real
embeddings, 48% smaller files and faster checkpoints and opens at 100K.
`clawhdf5-cli create --f32` opts out; like `--f32-index`, it only ever
switches the default off.
- `clawhdf5-agent`: `MemoryConfig::quantized_index` defaults to `true` for new
stores. The reason it had been off — that int8 search was slower on ARM —
did not survive measurement (see Corrections). Stores that predate the