feat(format): read unlimited and printf-style VDS mappings like libhdf5
Unlimited VDS mappings were refused, and printf-style source names
("f-%b.h5") were not expanded, so those regions read as fill (read-matrix
case 0470: 29 of 30 values wrong). All 7 virtual datasets in the libhdf5
test set use such mappings.
Implement H5Dvirtual.c's semantics in the vds module:
- %b is the block number, %% a literal %, other specifiers are an error;
block j of the virtual selection comes from the source named with j,
probing from 0 to the first missing source (printf gap 0);
- unlimited source/virtual selections are clipped to what the source's
current extent fills (H5S_hyper_get_clip_extent_match, partial last
block included);
- the extent is recomputed as H5Dget_space does (view "last available":
the largest clip, never below what limited mappings need), exposed as
vds::virtual_dataset_extent and used by Dataset::shape();
- a source in the other byte order is byte-swapped; other conversions stay
an error.
Tests: vds_interop::vds_printf_source_names,
vds_unlimited_mappings_follow_source_extents (h5py low-level API, earliest
and latest format) and vds_libhdf5_test_files (vds-eiger, 4_vds and
vds-percival-unlim-maxmin from HDF5's tools/test/testfiles/vds, committed
as fixtures) all compare shape and values with h5py; unit tests for the
clip arithmetic, name parsing and mapping rules.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -290,6 +290,19 @@
|
||||
**Behaviour change:** the raw-read API (`read_raw_data_full*`), which has
|
||||
no fill value, now returns an error for a virtual dataset with unmapped
|
||||
elements instead of zeros.
|
||||
- Unlimited and printf-style mappings are supported (all 7 VDS files in the
|
||||
libhdf5 test set are such mappings, e.g. Eiger/Percival detector layouts).
|
||||
`%b` in a source file or dataset name is the block number and `%%` a
|
||||
literal `%` (other `%` sequences are an error, as in libhdf5); block `j`
|
||||
is read from the source named with `j`, probing from 0 up to the first
|
||||
missing source. Unlimited source/virtual selections cover as much as the
|
||||
source's current extent fills, including a partial last block. As
|
||||
libhdf5 does on `H5Dget_space`, the extent is recomputed from the sources
|
||||
present (default "last available" view, printf gap 0) —
|
||||
`vds::virtual_dataset_extent`, used by `Dataset::shape()` — so e.g.
|
||||
`vds-eiger.h5` is `[5, 10, 10]`, not its stored `[20, 10, 10]`. A source
|
||||
stored in the other byte order is byte-swapped (libhdf5 converts);
|
||||
other type conversions remain an error.
|
||||
- Hyperslab selection versions 1 and 2 were refused ("only version-3
|
||||
hyperslab selections are supported"). Version 1 is what libhdf5 writes for
|
||||
every VDS created with the default format bounds (h5py's default), so
|
||||
|
||||
Reference in New Issue
Block a user