Commit Graph
5 Commits
Author SHA1 Message Date
osobhandClaude Fable 5.1 b55b7dbac5 fix(format): parse HDF5 2.0 native complex datatypes (class 11)
Class 11 (datatype version 5) properties are a single base floating-point
datatype message, not a compound-style member list. The old parser read the
base type's bytes as member names, yielding a garbage datatype, and failed
with UnexpectedEof when a complex type was nested in a compound.

Parse the base type and surface the type as the equivalent {r, i} compound
(the shape h5py writes for numpy complex dtypes), with a size check against
the base type. Covered by byte-level tests taken from HDF5 2.0 output and an
h5py end-to-end test (writer_h5py_tests is now 27/27 against HDF5 2.0.0).

Found while validating a user report of InvalidDatatypeVersion
{ class: 6, version: 5 } against v2.1.0 (already fixed on main in a13ff51,
never released). Add docs/known-issues.md recording that report, this bug,
the open reference-v4 gap and a gpu_tests parallel-run hang; credit the
reporter in the changelog.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-18 20:58:29 -07:00
Omar SobhandClaude Sonnet 5 88195d1c33 docs: fix untraceable benchmark claims, add dual-audience framing, validate on second machine
- README's "HDF5 Core I/O" table claimed 19ns/2,080µs labeled 308× (real ratio
  ~109,000×) and a 313ns zero-copy mmap figure — neither traced to any dated
  benchmark in BENCHMARKS.md. Replaced the table wholesale with the existing
  "vs libhdf5 Summary" figures, relabeled from "h5py/C HDF5" to "libhdf5"
  (BENCHMARKS.md never benchmarks against h5py, only libhdf5 directly).
- Added two new Criterion benchmarks to close the coverage gaps that produced
  the untraceable numbers: metadata_open_from_disk (I/O-inclusive, fair
  clawhdf5-vs-libhdf5 file-open comparison) and metadata_parse_in_memory
  (clawhdf5-only, explicitly labeled as excluding I/O) in h5bench_meta.rs;
  read_zerocopy_mmap in h5bench_read.rs (forces real page-ins by summing
  elements rather than just returning a slice length — the mmap path turns
  out to be slower than a plain copy at these sizes, an honest, unflattering
  but real result now documented instead of a fabricated 313ns).
- Re-ran the full existing benchmark suite plus the two new ones on a second,
  independently administered machine (tank: Ryzen 7 7800X3D) to validate the
  numbers before publishing them. 5 of 6 rows landed within ~15% of the
  original i7-12650H figures; recorded both in BENCHMARKS.md's new
  "Independent Validation" section. README now cites the tank numbers.
- Added a short top-of-file README callout naming both halves of the project
  (general-purpose HDF5 library vs. agent memory layer) with links to
  BENCHMARKS.md and the Crate Map, so a data-infra reader isn't 60% through
  a memory-store pitch before finding the part relevant to them.
- Added one factual, no-names line noting benchmark numbers are being
  validated in collaboration with HDF5 Group engineers.
- Fixed the same untraceable "2-300x faster than h5py/C HDF5" / "313 ns"
  claims in docs/QUICKSTART.md, one click from the README's own "New here?"
  link.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-08-03 17:46:55 -07:00
Omar SobhandClaude Sonnet 5 6b1ea450f5 chore: cleanup pass — remove empty types stub, implement superblock v4, reconcile plan docs
- Remove clawhdf5-types (empty 1-line stub crate; type defs already live in
  clawhdf5-format). Update workspace Cargo.toml and CLAUDE.md accordingly.
- Implement HDF5 superblock v4 (page-buffer mode) read and write support in
  clawhdf5-format: Superblock::parse_v4, page_size field, v4 serialize
  branch, and FileWriter::with_page_size. This was the one task left
  unimplemented from docs/superpowers/plans/2026-06-29-format-write-extensions.md.
- Reconcile the three docs/superpowers/plans/*.md docs (filter codecs,
  format write extensions, MPI-IO VOL) against actual shipped code: they
  were pre-work plans for d6c4d4f (2026-06-30) committed to git late on
  2026-08-03 with all checkboxes still unchecked. Mark completed tasks done
  and add a status note so they read as historical records, not open work.
- Refresh ROADMAP.md's "What's Next" section against current repo state.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-08-03 08:11:31 -07:00
Omar Sobh b1fc23e975 docs: add superpowers implementation plans (MPI-IO VOL backend, format write extensions, filter codecs) 2026-08-03 02:46:23 +00:00
redclawsystems 3f222f6956 Merge pull request 'docs(clawhdf5): document DType variants, fix unresolved doc links' (#17) from sdlc-docs/clawhdf5-types-20260514-165210 into main 2026-05-14 23:54:48 +00:00