Feat/pure rust default and msrv #3

2026-09-23T16:19:32Z

osobh commented

2026-09-23 16:19:32 +00:00

No description provided.

osobh added 3 commits 2026-09-23 16:19:33 +00:00

docs: bring the README up to date with the last five releases e9b37a9602

The README had fallen behind v2.3.0-v2.7.0, and parts of it were not
true. Checked every claim against the code and BENCHMARKS.md:

- Three of the six Quick Start snippets no longer compiled (Agent
  Memory, Consolidation, OpenClaw); all six now do.
- Hybrid search was described as RRF throughout. The default has been
  weighted 0.4/0.6 fusion since v2.5.0; re-ranking and confidence
  rejection run only in the OpenClaw backend.
- The `float16` feature does not halve embedding storage (the store
  always writes f32), `--features agent` enables nothing, "Source
  Isolation" is not wired in, and nothing backs "billion-scale" IVF-PQ.
- "Cryptographically verifiable" overstated an unkeyed, session-scoped
  FNV-1a ledger; "Zero C dependencies" was false while zlib-ng was the
  default deflate backend.
- Stale numbers: tests (1,650 -> 1,868), Rust badge (1.75 is below
  edition 2024's floor), 6.5 KB/record on disk (BENCHMARKS.md: 1.7 KB),
  consolidation and hybrid-search latency, and a feature-flag table
  broken by a paragraph pasted into it.
- The file schema, module table and crate map now match the code.

Adds a "What's new (v2.2 -> v2.7)" section for collaborators, leading
with the silent Extensible Array read bug fixed in v2.7.0. Footer links
point at git.redclaw.dev. CLAUDE.md: clawhdf5-migrate is the SQLite
migration tool, and MemoryConfig::compression is off by default.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

build: pure-Rust zlib-rs as the default deflate backend 735db117a7

The core crates (clawhdf5, -agent, -format, -io, -filters, -ann, -accel,
-netcdf4, -cli) now build no C by default: deflate defaults to zlib-rs,
a pure-Rust port of zlib-ng, and zlib-ng becomes the opt-in
`fast-deflate`, which overrides zlib-rs wherever it is enabled. A default
build no longer needs cmake or a C compiler.

Measured on tank, both builds run alternately, three rounds, medians:
zlib-rs is within 6% of zlib-ng on every HDF5 read and write (512x512
deflate-6 chunked write 1.458 vs 1.484 ms; 64 MB compressed read 64.4
vs 65.2 ms), and compressed output is byte-identical. Details in
BENCHMARKS.md, "Deflate backend".

Getting there took two fixes the first measurement exposed:

- zlib-rs needs `std` to detect SIMD at runtime. flate2 enables it via
  its default `runtime_detection`, which `default-features = false` had
  switched off, leaving zlib-rs 3.5x slower on inflate. The `zlib-rs`
  features now enable it.
- Both deflate paths streamed through flate2's 32 KiB read/write
  wrappers. They now hand the codec the whole chunk in one call, into a
  buffer sized up front (~5% on chunked writes). This also fixes a
  silent short read: the streaming reader returned a truncated stream's
  bytes without an error; a truncated chunk is now DecompressionError.
  In clawhdf5-filters, output longer than the stated size is now an
  error rather than silently cut off.

CI: ci-test.sh lints and tests the zlib-ng path, and fails if a
C-building crate (*-sys, cc, cmake) enters a core crate's default
dependency tree. The arm64 job no longer installs cmake.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

build: declare Rust 1.92 as the MSRV and check it in CI

CI / test-arm64 (pull_request) Successful in 1m19s

Details

CI / test (pull_request) Successful in 5m23s

Details

fc7ae6549a

rust-version = "1.92" in [workspace.package], inherited by every crate.
1.92 is the floor: wgpu (clawhdf5-gpu) requires it, and the whole
workspace, Python bindings included, checks cleanly on it. ci-test.sh
reads the version from Cargo.toml and checks the workspace on exactly
that toolchain, so the manifests and the README badge cannot drift from
what actually builds. The badge said 1.75, below edition 2024's own
floor.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

osobh merged commit 1cceb930b2 into main

2026-09-23 16:29:33 +00:00

osobh deleted branch feat/pure-rust-default-and-msrv

2026-09-23 16:29:36 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: quantumclaw/clawhdf5#3