Files
clawhdf5/crates
osobhandClaude Opus 5.5 5935e13866 fix(format): decode SZIP chunks the way libhdf5 does
SZIP-filtered datasets from libhdf5 came back as garbage or zeros with no
error (ref_szip.h5, h5repack_szip.h5, noencoder.h5, le_data/be_data
Szip_float_data_*), and 64-bit ones failed with "invalid bits per
sample" (h5wasm compressed.h5). The decoder called aec_buffer_decode
directly, but libhdf5 goes through szlib's SZ_BufftoBuffDecompress
(H5Zszip.c), which libaec implements with reshaping (sz_compat.c).
Differences, all fixed:

- H5Zszip.c prefixes the stream with the 4-byte LE uncompressed size; it
  was fed to libaec as data.
- 32- and 64-bit samples are coded as byte planes of 8-bit samples and
  must be de-interleaved.
- The reference sample interval is ceil(pixels_per_scanline /
  pixels_per_block), not a fixed 128.
- Scanlines that are not a whole number of blocks are padded and must be
  unpadded.
- Byte order comes from the MSB option bit; LE data was decoded as MSB.

Test: szip_decodes_libhdf5_chunks_exactly compares chunks from HDF Group
test files (noencoder.h5, le_data.h5) and an h5py-written file (64-bit,
16-bit, padded scanlines, NN and EC) byte for byte with h5py's values;
it failed before on the first case.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-25 21:11:24 -05:00
..