feat(agent): new stores default to float16 embeddings
MemoryConfig::float16 now defaults to true for new stores, on measurement: on the full LongMemEval haystack with real MiniLM embeddings every retrieval metric matched f32 (previous commit), and at 100K the file is 48% smaller with faster checkpoints and opens. Existing stores are unaffected: every agent store has recorded `float16 = false` in /meta and keeps it. A test opens the v2.5.0 fixture, saves and checkpoints, and checks the embeddings are still f32 with the old rows bit-identical; another checks a new store is float16. CLI: `create --f32` opts out; like `--f32-index` it only ever switches the default off. `--float16` is still accepted and now a no-op. Values beyond +-65504 are refused, so f32 remains the choice for unnormalised vectors — the upgrade note says so. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
+17
-6
@@ -11,12 +11,18 @@
|
||||
each checkpoint, so it becomes readable at its next checkpoint on this
|
||||
version; other files with `f32` or empty datasets need rewriting. Details in
|
||||
`docs/known-issues.md`.
|
||||
- **`MemoryConfig::float16` now does what it says.** It was persisted and
|
||||
otherwise ignored; embeddings were always stored as `f32`. A store created
|
||||
with it on now writes half-precision embeddings (48% smaller files) and
|
||||
rounds embeddings to half precision as they are saved. A store that already
|
||||
had `float16 = true` rounds its embeddings when next opened and writes them
|
||||
as `float16` at its next checkpoint. Off by default.
|
||||
- **New stores store embeddings as half precision by default.**
|
||||
`MemoryConfig::float16` was persisted and otherwise ignored; it now writes
|
||||
`float16` embeddings (48% smaller files at 100K) and rounds each embedding
|
||||
to half precision as it is saved — and it defaults to `true` for new
|
||||
stores. On the full LongMemEval haystack with real MiniLM embeddings every
|
||||
retrieval metric matched `f32`. **Existing stores are unaffected**: every
|
||||
agent store has recorded `float16 = false`, and keeps it (a v2.5.0 fixture
|
||||
guards this). A store that already had `float16 = true` rounds its
|
||||
embeddings when next opened and writes them as `float16` at its next
|
||||
checkpoint. Opt out with `float16 = false` or `create --f32`; the CLI's
|
||||
`--float16` is still accepted and now a no-op. Values beyond ±65504 are
|
||||
refused, so keep `f32` for unnormalised vectors.
|
||||
- **Breaking:** `MemoryError` gained `InvalidEntry`, returned when a
|
||||
`float16` store is given an embedding value beyond ±65504. Exhaustive
|
||||
matches need the new arm.
|
||||
@@ -123,6 +129,11 @@
|
||||
now an error.
|
||||
|
||||
### Defaults
|
||||
- `clawhdf5-agent`: `MemoryConfig::float16` defaults to `true` for new stores,
|
||||
measured rather than assumed: identical LongMemEval retrieval on real
|
||||
embeddings, 48% smaller files and faster checkpoints and opens at 100K.
|
||||
`clawhdf5-cli create --f32` opts out; like `--f32-index`, it only ever
|
||||
switches the default off.
|
||||
- `clawhdf5-agent`: `MemoryConfig::quantized_index` defaults to `true` for new
|
||||
stores. The reason it had been off — that int8 search was slower on ARM —
|
||||
did not survive measurement (see Corrections). Stores that predate the
|
||||
|
||||
Reference in New Issue
Block a user