fix: apply the superblock extension and cache image in every opener
File, MmapFile and LazyFile decoded the superblock extension and laid a metadata cache image over the file's metadata; the other readers did not, so the same file read differently by entry point: NativeVol, AsyncHDF5File and MpiVol (clawhdf5-io) and the external source files of a virtual dataset (clawhdf5-format vds.rs) read a file with an image from its own bytes, which libhdf5 does not (they may be stale, or zeros: h5clear_mdc_image.h5 failed with InvalidObjectHeaderVersion(0)), and skipped the extension checks File::open makes (cve-2020-10810/10812). Each of them owns its buffer, so each now calls the shared superblock_ext::apply_cache_image_in_place, which checks the extension and writes the image's entries in place (only the image block is copied). These readers read whole datasets and cannot open a file and fail each object, so an image libhdf5 cannot load is refused with the image's error, never read around. clawhdf5-io's vol::load_hdf5 wraps it for NativeVol (at open; for from_bytes the error is reported on read, as a truncated file already was) and MpiVol. The MpiVol edit is minimal and was not compiled: the mpi-io feature needs an MPI installation this machine does not have (mpi-sys's build script panics). Tests: NativeVol (open_path and from_bytes), AsyncHDF5File and a VDS whose source file is h5clear_mdc_image.h5 (vds_interop.rs, against h5py) read the fixture's values; the corrupted-image variants are refused. Each fails without its fix. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -486,3 +486,26 @@ fn vds_libhdf5_test_files() {
|
||||
vec![5, 10, 10]
|
||||
);
|
||||
}
|
||||
|
||||
/// A source file with a metadata cache image (libhdf5's
|
||||
/// `h5clear_mdc_image.h5`, whose root group's header exists only in the
|
||||
/// image) is read through the image, as libhdf5 opens it. Source files were
|
||||
/// read from their own bytes, and this one failed on the root group.
|
||||
#[test]
|
||||
fn vds_source_file_with_a_metadata_cache_image() {
|
||||
skip_if_no_python!();
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let fixture = Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/h5clear_mdc_image.h5");
|
||||
std::fs::copy(&fixture, dir.path().join("src.h5")).unwrap();
|
||||
generate(
|
||||
dir.path(),
|
||||
r#"
|
||||
with h5py.File("vds.h5", "w", libver="latest") as f:
|
||||
lay = h5py.VirtualLayout(shape=(3, 100), dtype="i4")
|
||||
lay[0:3, :] = h5py.VirtualSource("src.h5", "DSET", shape=(50, 100))[5:8, :]
|
||||
f.create_virtual_dataset("v", lay)
|
||||
expect("vds.h5", "v", "image_source")
|
||||
"#,
|
||||
);
|
||||
assert_matches_libhdf5(dir.path(), "vds.h5", "v", "image_source");
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user