feat(agent): new stores default to float16 embeddings
CI / test-arm64 (pull_request) Successful in 1m9s
CI / test (pull_request) Successful in 5m27s

MemoryConfig::float16 now defaults to true for new stores, on
measurement: on the full LongMemEval haystack with real MiniLM
embeddings every retrieval metric matched f32 (previous commit), and at
100K the file is 48% smaller with faster checkpoints and opens.

Existing stores are unaffected: every agent store has recorded
`float16 = false` in /meta and keeps it. A test opens the v2.5.0
fixture, saves and checkpoints, and checks the embeddings are still f32
with the old rows bit-identical; another checks a new store is float16.

CLI: `create --f32` opts out; like `--f32-index` it only ever switches
the default off. `--float16` is still accepted and now a no-op.
Values beyond +-65504 are refused, so f32 remains the choice for
unnormalised vectors — the upgrade note says so.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-24 19:39:33 -05:00
co-authored by Claude Opus 5.5
parent dbaf3f505d
commit 5c8323cb1e
6 changed files with 109 additions and 25 deletions
+14 -6
View File
@@ -36,10 +36,13 @@ enum Commands {
/// Accepted for compatibility; int8 is now the default
#[arg(long, hide = true, conflicts_with = "f32_index")]
quantized_index: bool,
/// Store embeddings on disk as IEEE half precision (float16): half
/// the bytes, about three significant digits; values must lie within
/// ±65504
/// Store embeddings as full-precision f32 instead of the default
/// half precision (float16: half the bytes, about three significant
/// digits, values within ±65504)
#[arg(long)]
f32: bool,
/// Accepted for compatibility; float16 is now the default
#[arg(long, hide = true, conflicts_with = "f32")]
float16: bool,
},
/// Save a memory entry (reads JSON from stdin or --json)
@@ -107,11 +110,16 @@ fn run(cli: Cli) -> Result<(), Box<dyn std::error::Error>> {
wal,
f32_index,
quantized_index: _,
float16,
f32,
float16: _,
} => {
let mut config = MemoryConfig::new(cli.path.clone(), &agent_id, dim);
config.wal_enabled = wal;
config.float16 = float16;
// As with --f32-index: only ever switch the library default off.
if f32 {
config.float16 = false;
}
let config_float16 = config.float16;
// Only ever switch *off* the library default: assigning the flag
// outright would force every CLI-created store back to f32 unless
// the caller knew to ask for int8.
@@ -127,7 +135,7 @@ fn run(cli: Cli) -> Result<(), Box<dyn std::error::Error>> {
"embedding_dim": dim,
"wal_enabled": wal,
"quantized_index": config_quantized,
"float16": float16,
"float16": config_float16,
"count": mem.count(),
});
println!("{}", serde_json::to_string_pretty(&j)?);