fix: apply the superblock extension and cache image in every opener

File, MmapFile and LazyFile decoded the superblock extension and laid a
metadata cache image over the file's metadata; the other readers did
not, so the same file read differently by entry point: NativeVol,
AsyncHDF5File and MpiVol (clawhdf5-io) and the external source files of
a virtual dataset (clawhdf5-format vds.rs) read a file with an image
from its own bytes, which libhdf5 does not (they may be stale, or zeros:
h5clear_mdc_image.h5 failed with InvalidObjectHeaderVersion(0)), and
skipped the extension checks File::open makes (cve-2020-10810/10812).

Each of them owns its buffer, so each now calls the shared
superblock_ext::apply_cache_image_in_place, which checks the extension
and writes the image's entries in place (only the image block is
copied). These readers read whole datasets and cannot open a file and
fail each object, so an image libhdf5 cannot load is refused with the
image's error, never read around. clawhdf5-io's vol::load_hdf5 wraps it
for NativeVol (at open; for from_bytes the error is reported on read,
as a truncated file already was) and MpiVol. The MpiVol edit is minimal
and was not compiled: the mpi-io feature needs an MPI installation this
machine does not have (mpi-sys's build script panics).

Tests: NativeVol (open_path and from_bytes), AsyncHDF5File and a VDS
whose source file is h5clear_mdc_image.h5 (vds_interop.rs, against
h5py) read the fixture's values; the corrupted-image variants are
refused. Each fails without its fix.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 11:49:13 -05:00
co-authored by Claude Opus 5.5
parent 6559a91495
commit d493d4792e
6 changed files with 176 additions and 6 deletions
+22 -1
View File
@@ -803,7 +803,11 @@ impl<'a, 'r> Sources<'a, 'r> {
let resolver = self.resolver.ok_or_else(|| {
vds_err("external-file virtual dataset sources require a file resolver")
})?;
self.cached_file = Some((String::from(name), resolver(name)?));
let mut bytes = resolver(name)?;
if let Some(b) = bytes.as_mut() {
load_source_file(b)?;
}
self.cached_file = Some((String::from(name), bytes));
}
// An external file is handed over whole; its addresses are relative
// to its superblock, so skip any user block.
@@ -851,6 +855,23 @@ impl<'a, 'r> Sources<'a, 'r> {
}
}
/// Check an external source file's superblock extension as libhdf5 does
/// when it opens the file, and write any metadata cache image over its
/// metadata in place: libhdf5 reads the image's entries instead of the
/// file's own, possibly stale, bytes (`crate::superblock_ext`). A source
/// file whose image cannot be loaded is an error, as other corrupt source
/// files are here.
fn load_source_file(whole: &mut [u8]) -> Result<(), FormatError> {
let base = crate::signature::find_signature(whole)?;
let sb = crate::superblock::Superblock::parse(&whole[base..], 0)?;
// The end of file the superblock records; a truncated source file is
// read as before, up to its length.
let end = sb
.data_end(base as u64, whole.len() as u64)
.map_or(whole.len(), |e| base + e as usize);
crate::superblock_ext::apply_cache_image_in_place(&mut whole[base..end], &sb)
}
/// Whether elements of `dt` contain addresses into their own file:
/// variable-length data (global-heap IDs) or references.
fn holds_file_addresses(dt: &Datatype) -> bool {