feat: integrate HNSW into agent search, fix Python 3.14 build
Resolves two gaps found in a project-state review:
1. Python build was broken: PyO3/numpy 0.23 caps at Python 3.13 but the
environment has 3.14. Bumped to 0.28 and updated the two breaking APIs
(PyObject -> Py<PyAny>, allow_threads -> detach). The extension module now
imports and round-trips under Python 3.14, unblocking cargo build --workspace.
2. The "HNSW vector search over agent memories" headline was unwired:
clawhdf5-ann had zero dependents and the agent used a linear cosine+BM25 scan.
- clawhdf5-ann is now a live index: insert, mark_deleted (soft delete with a
deleted bitset, traversed but never returned), compact, and a format
version tag (v2) with backward-compatible load of v1 files.
- clawhdf5-agent wires HNSW behind the `hnsw` feature (ON by default). The
index mirrors the cache (node id == cache index) and self-heals: it rebuilds
whenever hnsw_synced_len drifts from cache.len(), so unhooked pushes can't
desync it. Non-indexable stores (no/zero-dim/mixed embeddings) and queries
whose dim doesn't match fall back to the exact linear scan.
- hybrid.rs gains merge_vector_keyword, shared by the linear and HNSW paths.
- tests/hnsw_integration.rs validates recall vs a brute-force oracle plus
insert/delete/batch behaviour.
Disable HNSW for exact search with `--no-default-features --features float16`.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
@@ -44,7 +44,7 @@ impl PyAttrs {
|
||||
|
||||
#[pymethods]
|
||||
impl PyAttrs {
|
||||
fn __getitem__(&self, py: Python<'_>, key: &str) -> PyResult<PyObject> {
|
||||
fn __getitem__(&self, py: Python<'_>, key: &str) -> PyResult<Py<PyAny>> {
|
||||
match &self.inner {
|
||||
AttrsInner::Read(map) => match map.get(key) {
|
||||
Some(val) => Ok(attr_value_to_py(py, val)),
|
||||
@@ -100,7 +100,7 @@ impl PyAttrs {
|
||||
}
|
||||
}
|
||||
|
||||
fn __iter__(&self, py: Python<'_>) -> PyResult<PyObject> {
|
||||
fn __iter__(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {
|
||||
let keys = self.keys(py)?;
|
||||
let iter = keys.call_method0(py, "__iter__")?;
|
||||
Ok(iter)
|
||||
@@ -112,7 +112,7 @@ impl PyAttrs {
|
||||
}
|
||||
|
||||
/// Return attribute names as a list.
|
||||
fn keys(&self, py: Python<'_>) -> PyResult<PyObject> {
|
||||
fn keys(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {
|
||||
let names: Vec<String> = match &self.inner {
|
||||
AttrsInner::Read(map) => map.keys().cloned().collect(),
|
||||
AttrsInner::Write(store) => store
|
||||
@@ -127,8 +127,8 @@ impl PyAttrs {
|
||||
}
|
||||
|
||||
/// Return attribute values as a list.
|
||||
fn values(&self, py: Python<'_>) -> PyResult<PyObject> {
|
||||
let vals: Vec<PyObject> = match &self.inner {
|
||||
fn values(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {
|
||||
let vals: Vec<Py<PyAny>> = match &self.inner {
|
||||
AttrsInner::Read(map) => map.values().map(|v| attr_value_to_py(py, v)).collect(),
|
||||
AttrsInner::Write(store) => store
|
||||
.lock()
|
||||
@@ -145,8 +145,8 @@ impl PyAttrs {
|
||||
}
|
||||
|
||||
/// Return attribute (key, value) pairs as a list of tuples.
|
||||
fn items(&self, py: Python<'_>) -> PyResult<PyObject> {
|
||||
let pairs: Vec<(String, PyObject)> = match &self.inner {
|
||||
fn items(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {
|
||||
let pairs: Vec<(String, Py<PyAny>)> = match &self.inner {
|
||||
AttrsInner::Read(map) => map
|
||||
.iter()
|
||||
.map(|(k, v)| (k.clone(), attr_value_to_py(py, v)))
|
||||
|
||||
Reference in New Issue
Block a user