osobhandClaude Opus 4.8 a24fcb8be4 feat: read paged Fixed Array chunk indexes ...

A filtered, fixed-dimension dataset with more than one Fixed Array
data-block page (>1024 chunks by default) previously failed with
"paged Fixed Array data blocks not yet supported".

Implement the paged data-block layout, reverse-engineered and validated
against an HDF5 2.0 file:
- after the FADB prefix: a page-init bitmap (one bit per page, MSB-first
  within each byte), a 4-byte checksum, then the pages;
- each page is a fixed full-size slot of page_nelmts elements plus a
  4-byte checksum, with only the final page shorter;
- uninitialized pages still occupy their slot (zero-filled), so the
  bitmap — not a 0xFF sentinel — marks a whole page unallocated.

Element parsing is factored into parse_fa_element, shared by the
non-paged and paged paths.

Tests: real-file integration test against a minimal 2-page gzip fixture
(v4_fixed_array_paged.h5) plus a synthetic unit test covering a
multi-byte/MSB-first bitmap, a skipped uninitialized page, and a short
final page.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

2026-06-03 17:41:31 +00:00

35 KiB

Raw History

View Raw