quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
98 Commits 11 Branches 2 Tags
clawmates/mission-01a00d0f-407d84f0
Commit Graph
6 Commits
Author SHA1 Message Date
Omar Sobh a319405ffc security: Tier 3 — Android JNI length validation, pyo3 bump, WAL caps
CI / test (push) Failing after 2s
Details 
- clawhdf5-android: validate embedding_len/query_embedding_len against
  the handle's configured embedding_dim (and reject null pointers)
  before constructing a slice via from_raw_parts in edgehdf5_save and
  edgehdf5_hybrid_search. Strengthen the # Safety docs to state the
  now-enforced invariant and its limits. Add unit tests covering
  mismatched length and null-pointer rejection.
- clawhdf5-py: bump pyo3/numpy 0.28 -> 0.29, clearing RUSTSEC-2026-0176
  (OOB read in PyList/PyTuple iterator) and RUSTSEC-2026-0177 (missing
  Sync bound on PyCFunction::new_closure). No source changes needed;
  confirmed via cargo audit that both advisories no longer appear.
- clawhdf5-agent/wal.rs: cap read_len_prefixed_str/read_embedding's
  length claims at a new MAX_WAL_FIELD_LEN (64 MiB) before allocating,
  so a corrupted/truncated WAL length field fails cleanly instead of
  attempting a huge allocation. Add regression tests for both.
- BENCHMARKS.md: add a top-of-file traceability note distinguishing the
  dated/hardware-cited/reproducible h5bench and tank-validation sections
  from the older sections that don't yet meet that bar.
 2026-08-05 12:10:49 -07:00
Omar Sobh 62595d5ac0 chore: Tier 2 quick wins — version skew, docs, cleanup, overflow-safe bounds
CI / test (push) Failing after 14s
Details 
- Fix version skew: clawhdf5-py (pyproject.toml 1.93.0 -> 2.1.0) and
  packages/clawhdf5-node (package.json 2.0.0 -> 2.1.0) were both behind
  the actual crate version.
- Correct stale ROADMAP.md claims: the TypeScript bridge already has a
  complete napi-rs package (not "no package.json"); CI/CD is now wired
  up via .gitea/workflows/ci.yml.
- Fix CLAUDE.md: clawhdf5-gpu uses wgpu with hand-written WGSL compute
  shaders, not CubeCL.
- chunked_read.rs: drop 12 unnecessary chunk_dimensions[..rank].to_vec()
  allocations — all three callees already accept &[u32].
- btree_v1.rs: add an overflow-safe ensure_len(data, offset, needed)
  helper (checked_add) and use it at the two plain-arithmetic bounds
  guards, closing a usize-overflow edge case reachable from a crafted
  near-usize::MAX B-tree offset. Add a regression test.
- Clarify that the integrity hashes in clawhdf5-agent/provenance.rs
  (FNV-1a) and clawhdf5-format/provenance.rs (SHA-256) are unkeyed and
  only detect accidental corruption, not tampering — doc-only change.
- README.md: document that the mpi-io feature's read/write paths are
  root-read+broadcast / gather-to-rank-0, not true collective I/O.
 2026-08-05 12:02:23 -07:00
osobhandClaude Opus 4.8 f1762f82a7 docs: fix stale package names, counts, and CLI subcommands 
Sweep of the docs after the v2.0.0 rename and recent changes:
- Per-crate READMEs (13 files): rename leftover rustyhdf5-*/edgehdf5-*
  package names and badges to clawhdf5-*, bump usage versions to 2.1.0.
- README: update stale test badge (417 -> 1500+), workspace stats
  (15 crates/72K -> 17 crates/84K), agent crate stats (20.7K/32 modules),
  and add the missing clawhdf5-napi and clawhdf5-bench crates to the tree.
- CLAUDE.md: correct the CLI subcommand list (inspect/dump/index/search ->
  the actual create/save/search/recall/stats/flush-wal/agents-md/export/snapshot).

No code changes. Verified there are zero todo!()/unimplemented!() macros and
no TODO/FIXME comments in the tree.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
 2026-06-03 11:17:04 +00:00
osobhandClaude Opus 4.8 a16e9ab8cb chore: bump workspace to 2.1.0 
Bumps [workspace.package] and all 17 crate package versions (and internal
path-dependency requirements) from 2.0.0 to 2.1.0 for a coordinated release.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
 2026-06-03 10:36:17 +00:00
osobhandClaude Opus 4.8 8f9dbd812c feat: integrate HNSW into agent search, fix Python 3.14 build 
Resolves two gaps found in a project-state review:

1. Python build was broken: PyO3/numpy 0.23 caps at Python 3.13 but the
   environment has 3.14. Bumped to 0.28 and updated the two breaking APIs
   (PyObject -> Py<PyAny>, allow_threads -> detach). The extension module now
   imports and round-trips under Python 3.14, unblocking cargo build --workspace.

2. The "HNSW vector search over agent memories" headline was unwired:
   clawhdf5-ann had zero dependents and the agent used a linear cosine+BM25 scan.
   - clawhdf5-ann is now a live index: insert, mark_deleted (soft delete with a
     deleted bitset, traversed but never returned), compact, and a format
     version tag (v2) with backward-compatible load of v1 files.
   - clawhdf5-agent wires HNSW behind the `hnsw` feature (ON by default). The
     index mirrors the cache (node id == cache index) and self-heals: it rebuilds
     whenever hnsw_synced_len drifts from cache.len(), so unhooked pushes can't
     desync it. Non-indexable stores (no/zero-dim/mixed embeddings) and queries
     whose dim doesn't match fall back to the exact linear scan.
   - hybrid.rs gains merge_vector_keyword, shared by the linear and HNSW paths.
   - tests/hnsw_integration.rs validates recall vs a brute-force oracle plus
     insert/delete/batch behaviour.

Disable HNSW for exact search with `--no-default-features --features float16`.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
 2026-06-03 07:10:48 +00:00
redclawsystems 3f222f6956 Merge pull request 'docs(clawhdf5): document DType variants, fix unresolved doc links' (#17) from sdlc-docs/clawhdf5-types-20260514-165210 into main 2026-05-14 23:54:48 +00:00