quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity

feat(agent): new stores default to float16 embeddings

Browse Source 
MemoryConfig::float16 now defaults to true for new stores, on
measurement: on the full LongMemEval haystack with real MiniLM
embeddings every retrieval metric matched f32 (previous commit), and at
100K the file is 48% smaller with faster checkpoints and opens.

Existing stores are unaffected: every agent store has recorded
`float16 = false` in /meta and keeps it. A test opens the v2.5.0
fixture, saves and checkpoints, and checks the embeddings are still f32
with the old rows bit-identical; another checks a new store is float16.

CLI: `create --f32` opts out; like `--f32-index` it only ever switches
the default off. `--float16` is still accepted and now a no-op.
Values beyond +-65504 are refused, so f32 remains the choice for
unnormalised vectors — the upgrade note says so.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-24 19:39:33 -05:00
co-authored by Claude Opus 5.5
parent dbaf3f505d
commit 5c8323cb1e
6 changed files with 109 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+12 -8
View File
@@ -89,7 +89,8 @@ breaking change, are in [CHANGELOG.md](CHANGELOG.md).
fix themselves at their next checkpoint. See
[docs/known-issues.md](docs/known-issues.md).
- `MemoryConfig::float16` now stores half-precision embeddings (it was
ignored): 48% smaller files at the same recall.
ignored), and is on by default for new stores: 48% smaller files, and
identical LongMemEval retrieval on real embeddings.
- `HDF5Memory::search` with `SearchOptions`: filter by source channel (exact
filtered top-k, never slower than unfiltered), and opt-in re-ranking and
confidence rejection, which used to be OpenClaw-only.
@@ -256,8 +257,9 @@ retrieval recall reported as QA accuracy typically overstates by 20–30 points.
| 10K | 17.0 MB | 1.7 KB | 2.7 MB (6.2x) |
| 100K | 169.8 MB | 1.7 KB | 26.9 MB (6.2x) |
With `MemoryConfig::float16` the embeddings take half the space: an agent
store of 100K × 384 records is 80.8 MiB instead of 154.0.
These figures are `f32` embeddings. New agent stores default to
`MemoryConfig::float16`, which halves them: 100K × 384 records take 80.8 MiB
instead of 154.0.
**In memory** — a store reopened from disk, 384-dim `f32`, measured with a
counting allocator ([BENCHMARKS.md § Memory footprint](BENCHMARKS.md#memory-footprint)):
@@ -610,12 +612,14 @@ setting existed keep their `f32` index; opt out for new stores with
`quantized_index = false` or `clawhdf5-cli create --f32-index`. See
[BENCHMARKS.md § Quantising the index copy](BENCHMARKS.md#quantising-the-index-copy-quantized_index).
`MemoryConfig::float16` (off by default; CLI `create --float16`) stores the
`MemoryConfig::float16` (**on by default** for new stores) stores the
embeddings on disk as IEEE half precision (numpy `float16`): at 100K × 384 the
file drops from 154 to 81 MiB, checkpoints and opens get faster, and vector
recall and search latency do not change. Embeddings are rounded as they are
saved, so the store searches the same before and after a reopen; values must
lie within ±65504. See
file drops from 154 to 81 MiB, checkpoints and opens get faster, and on the
full LongMemEval haystack with real MiniLM embeddings every retrieval metric
matches `f32`. Embeddings are rounded as they are saved, so the store searches
the same before and after a reopen; values must lie within ±65504. Existing
stores keep their setting. Opt out with `float16 = false` or
`clawhdf5-cli create --f32` — e.g. for unnormalised vectors. See
[BENCHMARKS.md § float16 embedding storage](BENCHMARKS.md#float16-embedding-storage-memoryconfigfloat16).
### `clawhdf5-format`