quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity

fix(format): read unmapped VDS elements as the virtual dataset's fill value

Browse Source 
Elements of a virtual dataset that no mapping supplies (unmapped regions,
a missing source file, a missing source dataset) read as 0 instead of the
fill value libhdf5 returns — silent wrong data for any VDS created with a
non-zero fillvalue (read-matrix cases 0471/0472: -1 and 7 read as 0). A
missing source dataset was an error; libhdf5 reads it as fill.

Move VDS assembly into a new vds module following H5Dvirtual.c:
vds::read_virtual_dataset takes the dataset's fill value and a
VdsFileResolver that can refuse a name, and reports how many elements were
unmapped. Sources are read with their own fill value, and a source whose
datatype differs from the virtual dataset's is an error (libhdf5 converts).
File passes the dataset's fill value, resolves source names against the
virtual file's directory, and refuses names that leave it with an error
instead of reading them as fill. read_selection on a VDS goes through the
same fill-aware path.

The raw-read API (read_raw_data_full*) has no fill value, so it now errors
for a VDS with unmapped elements instead of guessing zeros.

Tests: vds_interop::vds_unmapped_regions_read_as_fill_value (external,
same-file, missing file/dataset, sparse source with its own fill, int
fill; earliest and latest format) and
vds_source_outside_directory_is_an_error_not_fill, both against h5py;
integration_test::v4_virtual_dataset_raw_api_refuses_to_guess_the_fill_value.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-25 22:03:47 -05:00
co-authored by Claude Opus 5.5
parent 2c6c6c176e
commit e94a52a88b
8 changed files with 785 additions and 167 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/known-issues.md
+5 -3
View File
@@ -63,13 +63,15 @@ segfault or abort.
## Gaps found by the 2026-09-25 HDF5 audit (open)
**Status:** open. These fail with an error; none returns wrong data, except
the VDS item, which is marked.
**Status:** open. These fail with an error; none returns wrong data (the VDS
fill-value item that did is fixed).
- **Layout message versions 1 and 2** (HDF5 1.6-era files): 84 of the 686
sweep files, `InvalidLayoutVersion`. This is the largest single gap.
- **Virtual datasets:**
- **Wrong data:** unmapped regions read as 0 instead of the fill value.
- ~~**Wrong data:** unmapped regions read as 0 instead of the fill value.~~
Fixed 2026-09-25: unmapped elements and missing sources read as the
virtual dataset's fill value.
- `%b` printf-style source names are not expanded.
- ~~Hyperslab selection versions 1 and 2 are refused.~~ Fixed 2026-09-25:
versions 1-3 and irregular hyperslabs are decoded.