clawhdf5

80 Commits 11 Branches 2 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
Omar Sobh	55959b4920	ci: wire up CI, fix no_std build, fix stale package names in scripts CI / test (push) Failing after 15s Details - Add .gitea/workflows/ci.yml running scripts/ci-test.sh (fmt, clippy, test, no_std check) on push/PR to main. - Fix stale rustyhdf5-py/rustyhdf5-format package names in ci-test.sh/check-nostd.sh, which had been silently no-op'ing those checks (cargo warns but doesn't fail on an unknown --exclude/-p target). - With those checks actually running, fix the real issues they surface: - clippy: useless_conversion in chunked_write.rs, byte_char_slices in global_heap.rs/object_header.rs. - cargo fmt: apply formatting across the workspace (whitespace only). - no_std (thumbv7em-none-eabihf) build errors in clawhdf5-format: core::sync::atomic::AtomicU64 doesn't exist on that target (no native 64-bit atomics) — switch profiling.rs's counters to portable-atomic, which falls back to a CAS-based emulation there and is a no-op wrapper elsewhere. Add missing alloc imports for Box (filters.rs), Vec (filters_szip.rs), and format! (dict_encoding.rs) on no_std paths. Replace f64::powi (std/libm-only) with a small local exponentiation-by-squaring helper in the scale-offset filter.	2026-08-05 10:50:13 -07:00
osobhandClaude Opus 4.8	8f9dbd812c	feat: integrate HNSW into agent search, fix Python 3.14 build Resolves two gaps found in a project-state review: 1. Python build was broken: PyO3/numpy 0.23 caps at Python 3.13 but the environment has 3.14. Bumped to 0.28 and updated the two breaking APIs (PyObject -> Py<PyAny>, allow_threads -> detach). The extension module now imports and round-trips under Python 3.14, unblocking cargo build --workspace. 2. The "HNSW vector search over agent memories" headline was unwired: clawhdf5-ann had zero dependents and the agent used a linear cosine+BM25 scan. - clawhdf5-ann is now a live index: insert, mark_deleted (soft delete with a deleted bitset, traversed but never returned), compact, and a format version tag (v2) with backward-compatible load of v1 files. - clawhdf5-agent wires HNSW behind the `hnsw` feature (ON by default). The index mirrors the cache (node id == cache index) and self-heals: it rebuilds whenever hnsw_synced_len drifts from cache.len(), so unhooked pushes can't desync it. Non-indexable stores (no/zero-dim/mixed embeddings) and queries whose dim doesn't match fall back to the exact linear scan. - hybrid.rs gains merge_vector_keyword, shared by the linear and HNSW paths. - tests/hnsw_integration.rs validates recall vs a brute-force oracle plus insert/delete/batch behaviour. Disable HNSW for exact search with `--no-default-features --features float16`. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>	2026-06-03 07:10:48 +00:00
redclawsystems	3f222f6956	Merge pull request 'docs(clawhdf5): document DType variants, fix unresolved doc links' (#17 ) from sdlc-docs/clawhdf5-types-20260514-165210 into main	2026-05-14 23:54:48 +00:00

Author

SHA1

Message

Date

Omar Sobh

55959b4920

ci: wire up CI, fix no_std build, fix stale package names in scripts

CI / test (push) Failing after 15s

Details

- Add .gitea/workflows/ci.yml running scripts/ci-test.sh (fmt, clippy,
  test, no_std check) on push/PR to main.
- Fix stale rustyhdf5-py/rustyhdf5-format package names in
  ci-test.sh/check-nostd.sh, which had been silently no-op'ing those
  checks (cargo warns but doesn't fail on an unknown --exclude/-p
  target).
- With those checks actually running, fix the real issues they surface:
  - clippy: useless_conversion in chunked_write.rs, byte_char_slices in
    global_heap.rs/object_header.rs.
  - cargo fmt: apply formatting across the workspace (whitespace only).
  - no_std (thumbv7em-none-eabihf) build errors in clawhdf5-format:
    core::sync::atomic::AtomicU64 doesn't exist on that target (no
    native 64-bit atomics) — switch profiling.rs's counters to
    portable-atomic, which falls back to a CAS-based emulation there
    and is a no-op wrapper elsewhere. Add missing alloc imports for
    Box (filters.rs), Vec (filters_szip.rs), and format! (dict_encoding.rs)
    on no_std paths. Replace f64::powi (std/libm-only) with a small
    local exponentiation-by-squaring helper in the scale-offset filter.

2026-08-05 10:50:13 -07:00

osobhandClaude Opus 4.8

8f9dbd812c

feat: integrate HNSW into agent search, fix Python 3.14 build

Resolves two gaps found in a project-state review:

1. Python build was broken: PyO3/numpy 0.23 caps at Python 3.13 but the
   environment has 3.14. Bumped to 0.28 and updated the two breaking APIs
   (PyObject -> Py<PyAny>, allow_threads -> detach). The extension module now
   imports and round-trips under Python 3.14, unblocking cargo build --workspace.

2. The "HNSW vector search over agent memories" headline was unwired:
   clawhdf5-ann had zero dependents and the agent used a linear cosine+BM25 scan.
   - clawhdf5-ann is now a live index: insert, mark_deleted (soft delete with a
     deleted bitset, traversed but never returned), compact, and a format
     version tag (v2) with backward-compatible load of v1 files.
   - clawhdf5-agent wires HNSW behind the `hnsw` feature (ON by default). The
     index mirrors the cache (node id == cache index) and self-heals: it rebuilds
     whenever hnsw_synced_len drifts from cache.len(), so unhooked pushes can't
     desync it. Non-indexable stores (no/zero-dim/mixed embeddings) and queries
     whose dim doesn't match fall back to the exact linear scan.
   - hybrid.rs gains merge_vector_keyword, shared by the linear and HNSW paths.
   - tests/hnsw_integration.rs validates recall vs a brute-force oracle plus
     insert/delete/batch behaviour.

Disable HNSW for exact search with `--no-default-features --features float16`.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

2026-06-03 07:10:48 +00:00

redclawsystems

3f222f6956

Merge pull request 'docs(clawhdf5): document DType variants, fix unresolved doc links' (#17 ) from sdlc-docs/clawhdf5-types-20260514-165210 into main

2026-05-14 23:54:48 +00:00