CHANGELOG.md

@@ -296,6 +296,57 @@
 - CI keeps zlib-ng building and tested; the arm64 job no longer needs cmake.
 
 ### Correctness
+- **Corrupt files libhdf5 refuses are now refused instead of read.** On the
+  HDF Group's CVE reproducers, 18 objects that libhdf5 (HDF5 2.0, through
+  h5py) refuses to open were read by clawhdf5, some as wrong data (a chunk
+  dimension of 0 read as all fill values; chunks read at offsets off the
+  chunk grid). The
+  parser now makes libhdf5's checks, with libhdf5's error text:
+  - object headers (`FormatError::InvalidObjectHeader`): every message of a
+    v1 chunk is read and more than the prefix's count is refused (the rest
+    used to be dropped); v1 message sizes must be multiples of 8 and a v1
+    chunk cannot end in a gap; a message running past its chunk is an error
+    (it used to end the chunk quietly); contradictory message flags; a
+    message of a class that cannot be shared flagged shareable; a
+    reference-count message in a v1 header; malformed continuation,
+    reference-count and modification-time messages; unknown v2 header
+    flags.
+  - datatypes (`FormatError::InvalidDatatype`): size 0; integer bits outside
+    the type; float exponent/mantissa outside the type, empty or
+    overlapping; a compound with no members, a member outside the compound,
+    a duplicate name or overlapping members; an enum whose size differs from
+    its base type's or with an empty name; array rank over 32 or a zero
+    dimension; an opaque tag length that is not a multiple of 8; in a
+    version-1 (unchecksummed) header, a numeric type that leaves more than
+    half its bits unused (`Datatype::parse_in_header`,
+    `Datatype::check_unused_bits`). A v1/v2 float's class bit 6 was read as
+    VAX byte order; libhdf5 ignores it before version 3, and so does this.
+  - chunked layouts (`FormatError::InvalidChunkDimensions`): a zero chunk
+    dimension, a chunk rank that does not match the dataspace, a chunk of
+    4 GiB or more (0x80000000-sized chunks hung the reader), and v1 B-tree
+    chunk keys whose offsets are not multiples of the chunk dimensions,
+    including the keys that only bound a node
+    (`chunked_read::collect_chunk_info_checked`).
+  - truncated files (`FormatError::TruncatedFile`, `Superblock::data_end`):
+    a file shorter than the end of file its superblock records is refused
+    ("truncated file"), and nothing past that end is read. `File`,
+    `LazyFile` and `MmapFile` do this.
+
+  Checks newer libhdf5 releases make but HDF5 2.0 does not (bit-field
+  offsets, the variable-length kind, array sizes) are left out, so files
+  h5py opens still open. Two libhdf5 checks are skipped on purpose because
+  clawhdf5 up to v2.7.0 wrote files that fail them without being wrong:
+  the sign bit of every float at position 63, and a size-0 string type for
+  an empty-string attribute (new fixtures written by v2.7.0 guard this).
+  Conformance: 569 -> 571 ok (h5stat_err_refcount.h5,
+  h5clear_fsm_persist_less.h5), and 17 of the 18 CVE objects now fail as in
+  libhdf5 (see `docs/known-issues.md` for the one left), as do 10 files
+  h5py refuses as truncated. Tests:
+  `header_validation_interop.rs` (h5py writes, the test damages a copy, both
+  libraries must refuse it), `legacy_writer_files.rs`, and unit tests next
+  to each check. **Breaking (format crate):** `FormatError` gained
+  `InvalidObjectHeader`, `InvalidDatatype`, `InvalidChunkDimensions` and
+  `TruncatedFile`; an exhaustive `match` on it needs the new arms.
 - `clawhdf5-format` VDS: variable-length and reference data from a source in
   another file is refused. Those elements are global-heap IDs and object
   addresses in the source file; copied into the virtual dataset they would

docs/known-issues.md

@@ -134,8 +134,30 @@ fill-value item that did is fixed).
   - N-Bit on 64-bit scale-offset data and some N-Bit parameter layouts fail.
 - **Filters:** blosc, blosc2, bitshuffle, bzip2, LZF and zfp are not
   implemented.
-- **Header checks:** on 12 CVE datasets libhdf5 rejects a corrupt header and
+- ~~**Header checks:** on 12 CVE datasets libhdf5 rejects a corrupt header and
-  we read data anyway. We need stricter header checks.
+  we read data anyway. We need stricter header checks.~~ **Fixed
+  2026-09-26** (counted again: 18 objects on the CVE corpus that libhdf5
+  refuses; some read as wrong data, e.g. a zero chunk dimension read as all
+  fill values): object headers, datatypes, chunk dimensions and chunk-index
+  offsets are checked as libhdf5 checks them, and truncated files are
+  refused. 17 of the 18 now
+  fail as in libhdf5 (conformance on tank, `conformance/run.sh --no-fetch`,
+  2026-09-26: 571 of 697 ok). Still read where libhdf5 refuses:
+  - `cve-2024-32624.h5` `/Dset_OBJREF`: a dataspace whose storage size
+    overflows 64 bits. `File::dataset` and `shape()` succeed (libhdf5
+    refuses at open); reading the values fails.
+  - `cve-2020-10810.h5`, `cve-2020-10812.h5` (whole files libhdf5 cannot
+    open, not among the 18): libhdf5 decodes the superblock extension's File
+    Space Info and metadata-cache-image messages at open and refuses these
+    files; we do not decode those messages at open.
+  - Deliberately not refused, because clawhdf5 up to v2.7.0 wrote them: a
+    float sign bit position outside the type, and a size-0 string type.
+  - Not refused because HDF5 2.0 (h5py 3.16) reads them though newer
+    libhdf5 refuses them: bit-field offset/precision outside the type, an
+    unknown variable-length kind, an array type whose stored size is not
+    its element count times its base size.
+  - (`cve-2024-32616` `/group1/dset3` and `cve-2025-2309`'s `Comp_OBJREF`
+    attribute are h5py/numpy type-mapping failures, not libhdf5 refusals.)
 - **Writer:**
   - Nested groups beyond one level: path-like names are now refused, not
     created.