feat(agent): MemoryConfig::float16 stores half-precision embeddings
CI / test-arm64 (pull_request) Successful in 1m19s
CI / test (pull_request) Successful in 4m58s

The setting was persisted in /meta and otherwise ignored: embeddings
were always written as f32. It now does what it says.

clawhdf5-format:
- `DatasetBuilder::with_f16_data` writes IEEE binary16 (numpy float16),
  rounding to nearest-even, and `make_f16_type`.
- `clawhdf5_format::float16` holds the f32 <-> f16 conversions, the one
  implementation the writer, the reader and the agent all use. Checked
  against the `half` crate on 16.7M f32 values and round-trips all 65536
  half values; the h5py interop tests confirm the rounding matches
  numpy's bit for bit (4020 values incl. ties, subnormals, overflow).
- Reading little-endian float16 as f32 has a fast path.

clawhdf5-agent:
- A float16 store writes /memory/embeddings as half precision, and
  `MemoryCache::half_precision` rounds each embedding as it enters the
  cache (save, update, WAL replay, and on load of a store still f32 on
  disk), so memory and file agree bit for bit and a store searches the
  same before and after a reopen (tested).
- Values beyond +-65504 are refused with the new
  `MemoryError::InvalidEntry` rather than stored as infinity, on every
  save path; batches are all or nothing, and a rejected ephemeral entry
  stays in the ephemeral tier. Breaking for exhaustive matches.
- CLI: `create --float16`. Off by default.

Measured on tank, 384-dim, six runs alternating order, medians
(search_harness --float16-study --full): at 100K the file goes from
154.0 to 80.8 MiB (-48%), checkpoint 752 -> 512 ms, open 300 -> 252 ms;
vector recall@10 against an exact scan and hybrid_search latency do not
change. At 10K open is 3 ms slower. Also a test that h5py opens a whole
agent store, f32 and float16, and decodes every dataset.

Docs: README, BENCHMARKS.md ("float16 embedding storage"), CHANGELOG
(including the h5py interop fixes in the previous commit), CLAUDE.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-24 12:00:38 -05:00
co-authored by Claude Opus 5.5
parent 5e4aa1c6bf
commit d0db83812b
18 changed files with 1111 additions and 43 deletions
+24 -3
View File
@@ -82,6 +82,15 @@ breaking change, are in [CHANGELOG.md](CHANGELOG.md).
100K × 384 store to 1.74× the raw vectors. At equal recall it is also faster
than `f32`: 1.63× QPS on AVX2, 1.18× on a Raspberry Pi 5 (NEON `SDOT`).
**Interop (unreleased)**
- **Files we write now open in h5py and libhdf5.** Every `f32` dataset —
including every agent store's embeddings — and every empty dataset was
refused by libhdf5. Both were write-side bugs in every release; agent stores
fix themselves at their next checkpoint. See
[docs/known-issues.md](docs/known-issues.md).
- `MemoryConfig::float16` now stores half-precision embeddings (it was
ignored): 48% smaller files at the same recall.
**Tooling**
- CI now runs the h5py/netCDF4 interop suites for real (they had been skipping
silently) and runs an aarch64 job for the NEON kernels.
@@ -244,6 +253,9 @@ retrieval recall reported as QA accuracy typically overstates by 20–30 points.
| 10K | 17.0 MB | 1.7 KB | 2.7 MB (6.2x) |
| 100K | 169.8 MB | 1.7 KB | 26.9 MB (6.2x) |
With `MemoryConfig::float16` the embeddings take half the space: an agent
store of 100K × 384 records is 80.8 MiB instead of 154.0.
**In memory** — a store reopened from disk, 384-dim `f32`, measured with a
counting allocator ([BENCHMARKS.md § Memory footprint](BENCHMARKS.md#memory-footprint)):
@@ -538,7 +550,7 @@ ClawhDF5's agent memory design draws from 15+ recent papers:
| Flag | Default | Description |
|------|---------|-------------|
| `float16` | **yes** | Half-precision cosine kernel (`cosine_similarity_f16`). The store itself always writes `f32` embeddings; `MemoryConfig::float16` is recorded in `/meta` but not yet applied |
| `float16` | **yes** | Half-precision cosine kernel (`cosine_similarity_f16`). Half-precision *storage* is the `MemoryConfig::float16` setting below, and needs no feature |
| `hnsw` | **yes** | HNSW approximate vector index for `hybrid_search` (via `clawhdf5-ann`); disable for an exact linear scan |
| `parallel` | **yes** | Parallel HNSW bulk build (same graph, ~3× faster on 16 cores) and Rayon brute-force search strategies |
| `zstd` | no | Compress embeddings with Zstd instead of deflate when `MemoryConfig::compression` is on (links libzstd) |
@@ -568,6 +580,14 @@ setting existed keep their `f32` index; opt out for new stores with
`quantized_index = false` or `clawhdf5-cli create --f32-index`. See
[BENCHMARKS.md § Quantising the index copy](BENCHMARKS.md#quantising-the-index-copy-quantized_index).
`MemoryConfig::float16` (off by default; CLI `create --float16`) stores the
embeddings on disk as IEEE half precision (numpy `float16`): at 100K × 384 the
file drops from 154 to 81 MiB, checkpoints and opens get faster, and vector
recall and search latency do not change. Embeddings are rounded as they are
saved, so the store searches the same before and after a reopen; values must
lie within ±65504. See
[BENCHMARKS.md § float16 embedding storage](BENCHMARKS.md#float16-embedding-storage-memoryconfigfloat16).
### `clawhdf5-format`
| Flag | Default | Description |
@@ -656,8 +676,9 @@ agent_memory.h5
│ └── ann_generation (ties the .ann sidecar to this checkpoint)
├── /memory
│ ├── chunks: string[N]
│ ├── embeddings: f32[N × D] (chunked; deflate, or Zstd with the
│ │ `zstd` feature, when compression is on)
│ ├── embeddings: f32[N × D], or f16 for a `float16` store
│ │ (chunked; deflate, or Zstd with the `zstd`
│ │ feature, when compression is on)
│ ├── source_channel: string[N]
│ ├── timestamps: f64[N]
│ ├── session_ids: string[N]