fix: apply the superblock extension and cache image in every opener

File, MmapFile and LazyFile decoded the superblock extension and laid a
metadata cache image over the file's metadata; the other readers did
not, so the same file read differently by entry point: NativeVol,
AsyncHDF5File and MpiVol (clawhdf5-io) and the external source files of
a virtual dataset (clawhdf5-format vds.rs) read a file with an image
from its own bytes, which libhdf5 does not (they may be stale, or zeros:
h5clear_mdc_image.h5 failed with InvalidObjectHeaderVersion(0)), and
skipped the extension checks File::open makes (cve-2020-10810/10812).

Each of them owns its buffer, so each now calls the shared
superblock_ext::apply_cache_image_in_place, which checks the extension
and writes the image's entries in place (only the image block is
copied). These readers read whole datasets and cannot open a file and
fail each object, so an image libhdf5 cannot load is refused with the
image's error, never read around. clawhdf5-io's vol::load_hdf5 wraps it
for NativeVol (at open; for from_bytes the error is reported on read,
as a truncated file already was) and MpiVol. The MpiVol edit is minimal
and was not compiled: the mpi-io feature needs an MPI installation this
machine does not have (mpi-sys's build script panics).

Tests: NativeVol (open_path and from_bytes), AsyncHDF5File and a VDS
whose source file is h5clear_mdc_image.h5 (vds_interop.rs, against
h5py) read the fixture's values; the corrupted-image variants are
refused. Each fails without its fix.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 11:49:13 -05:00
co-authored by Claude Opus 5.5
parent 6559a91495
commit d493d4792e
6 changed files with 176 additions and 6 deletions
+41
View File
@@ -288,6 +288,12 @@ impl AsyncHDF5File {
// superblock records, as libhdf5 does.
let end = superblock.data_end(user_block as u64, whole_len)?;
data.truncate(end as usize);
// Check the superblock extension as libhdf5 does at open, and write
// any metadata cache image over the file's metadata (libhdf5 reads
// the image's entries instead of the file's own, possibly stale,
// bytes). An image libhdf5 cannot load is refused: this reader has
// no way to open the file and fail each object instead.
clawhdf5_format::superblock_ext::apply_cache_image_in_place(&mut data, &superblock)?;
Ok(Self { data, superblock })
}
@@ -430,6 +436,41 @@ mod tests {
fw.finish().unwrap()
}
/// libhdf5's `h5clear_mdc_image.h5` (see the clawhdf5 crate's
/// `tests/metadata_cache_image.rs`): the root group's header exists
/// only in the file's metadata cache image.
fn cache_image_fixture() -> Vec<u8> {
std::fs::read(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../clawhdf5/tests/fixtures/h5clear_mdc_image.h5"
))
.unwrap()
}
#[tokio::test]
async fn reads_through_a_metadata_cache_image() {
let bytes = cache_image_fixture();
let file = AsyncHDF5File::from_bytes(bytes.clone()).unwrap();
let info = file.read_dataset_raw("DSET").await.unwrap();
assert_eq!(info.shape, [50, 100]);
let values: Vec<i32> = info
.raw
.chunks_exact(4)
.map(|b| i32::from_le_bytes(b.try_into().unwrap()))
.collect();
let expected: Vec<i32> = (0..50).flat_map(|i| (0..100).map(move |j| i * j)).collect();
assert_eq!(values, expected);
// An image libhdf5 cannot load is refused at open.
let mut bad = bytes;
let at = bad.windows(4).position(|w| w == b"MDCI").unwrap();
bad[at] = b'X';
assert!(matches!(
AsyncHDF5File::from_bytes(bad),
Err(AsyncHDF5Error::Format(FormatError::InvalidCacheImage(_)))
));
}
// --- AsyncMemoryReader tests ---
#[tokio::test]