quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
196 Commits 1 Branch 12 Tags
4a5544da1d223a1249df0794de6aff920f1323d1
Commit Graph
6 Commits
Author SHA1 Message Date
osobhandClaude Opus 5 97e65f2adf feat(accel): runtime-dispatched int8 dot product 
The int8-quantised HNSW index compared vectors with a scalar loop that
the compiler vectorised for the x86-64 baseline (SSE2), while the f32
path it was measured against goes through `clawhdf5-accel` and runs
AVX2. So the ~13% throughput cost recorded for `quantized_index` was a
missing kernel rather than a property of int8.

`clawhdf5_accel::dot_i8` adds a scalar fallback and an AVX2 path:
sign-extend each 16-byte half to i16, then `madd_epi16`, which
multiplies and sums adjacent pairs straight into i32 lanes. It is
dispatched through the same detected backend as the f32 kernels, and
the index now calls it.

Integer arithmetic, so the SIMD path must agree with scalar bit for
bit — tested at lengths that are and are not multiples of the block,
and at the -128 extreme for overflow.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
 2026-09-20 17:30:56 -07:00
osobhandClaude Fable 5.1 f15bf2eb22 perf(ann): unit-vector dot product and a reusable visited set 
- Cosine distance was 1 - dot/(|a||b|), re-deriving both norms on every
  evaluation in the innermost loop of build and search. The index now stores
  unit vectors (prepared at build, insert, graph load and HDF5 load; the query
  once per search) and uses 1 - dot. Zero vectors stay zero, giving distance 1
  as before. Returned distances are unchanged.
- search_layer allocated a HashSet of visited nodes per call. It is now an
  epoch-stamped u32 array in thread-local scratch, reused across calls, so
  search(&self) stays shareable between threads.
- clawhdf5-accel caches the detected SIMD backend in a OnceLock.

Recall is identical. Build 2.75 -> 1.89 s (10K), ~38 -> 21 s (100K); QPS at
ef=64 22.7K -> 39K (10K), 10.4K -> 14K (100K).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
 2026-09-19 13:21:40 -07:00
osobhandClaude Fable 5.1 a3f7c6fe89 style: cargo fmt --all 
Formatting only. cargo fmt --check was already failing on main (accel SIMD
kernels, agent, format, migrate, bench); CI now enforces it.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
 2026-09-19 05:36:23 -07:00
clawhdf5 committer agentandClaude Sonnet 5 377c8b6f17 fix(accel): restore f32::EPSILON near-zero-denom guard in cosine_similarity
CI / test (pull_request) Canceled after 0s
Details 
The SIMD migration weakened the near-zero-norm guard in all four
clawhdf5-accel cosine_similarity backends (scalar/avx2/avx512/neon)
from `denom < f32::EPSILON` to `denom == 0.0`. Vectors with a tiny
but nonzero norm (denom in (0, 1.19e-7)) fell through to dot/denom
and scored as identical instead of maximally dissimilar, diverging
from the pre-SIMD scalar loop's documented fallback behavior.

Restores the epsilon threshold in all four backends so
`1.0 - cosine_similarity(...)` in hnsw.rs::compute_distance
reproduces the old fallback exactly. Adds regression tests in
clawhdf5-accel and clawhdf5-ann locking in the near-zero-norm case.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
 2026-08-17 13:28:43 +00:00
Omar Sobh 55959b4920 ci: wire up CI, fix no_std build, fix stale package names in scripts
CI / test (push) Failing after 15s
Details 
- Add .gitea/workflows/ci.yml running scripts/ci-test.sh (fmt, clippy,
  test, no_std check) on push/PR to main.
- Fix stale rustyhdf5-py/rustyhdf5-format package names in
  ci-test.sh/check-nostd.sh, which had been silently no-op'ing those
  checks (cargo warns but doesn't fail on an unknown --exclude/-p
  target).
- With those checks actually running, fix the real issues they surface:
  - clippy: useless_conversion in chunked_write.rs, byte_char_slices in
    global_heap.rs/object_header.rs.
  - cargo fmt: apply formatting across the workspace (whitespace only).
  - no_std (thumbv7em-none-eabihf) build errors in clawhdf5-format:
    core::sync::atomic::AtomicU64 doesn't exist on that target (no
    native 64-bit atomics) — switch profiling.rs's counters to
    portable-atomic, which falls back to a CAS-based emulation there
    and is a no-op wrapper elsewhere. Add missing alloc imports for
    Box (filters.rs), Vec (filters_szip.rs), and format! (dict_encoding.rs)
    on no_std paths. Replace f64::powi (std/libm-only) with a small
    local exponentiation-by-squaring helper in the scale-offset filter.
 2026-08-05 10:50:13 -07:00
redclawsystems 3f222f6956 Merge pull request 'docs(clawhdf5): document DType variants, fix unresolved doc links' (#17) from sdlc-docs/clawhdf5-types-20260514-165210 into main 2026-05-14 23:54:48 +00:00