test: regression for scale-offset float E-scale (raw + masked filter)

The HDF5 library does not implement the scale-offset filter's floating-point
E-scale mode. When asked for it (cd_values[0] = 1) it stores the chunk raw
(no minbits/minval header) and sets the chunk filter mask to skip the filter,
so such datasets read back verbatim purely by honoring the per-chunk filter
mask — no E-scale decoder is required.

Add a fixture written via the HDF5 low-level API (exact-representable values)
and a test asserting it reads back verbatim, locking in the filter-mask path.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
osobh
2026-06-03 22:14:01 +00:00
co-authored by Claude Opus 4.8
parent e6f0d8f161
commit 57adc88320
3 changed files with 22 additions and 0 deletions
+8
View File
@@ -39,6 +39,14 @@
error. The previous `parse_vds_mappings` used a guessed layout that did not error. The previous `parse_vds_mappings` used a guessed layout that did not
match real files and is replaced. match real files and is replaced.
### Tests
- `clawhdf5-format`: regression test for **scale-offset float E-scale**
datasets. The HDF5 library does not implement E-scale encoding — when asked
for it (`cd_values[0] = 1`) it stores the chunk raw and sets the chunk filter
mask to skip the filter — so these files read back verbatim purely by
honoring the per-chunk filter mask. The test locks in that behavior against a
fixture produced via the HDF5 low-level API; no E-scale decoder is needed.
### Bug Fixes ### Bug Fixes
- `clawhdf5-format`: read **paged Fixed Array** chunk indexes. A filtered, - `clawhdf5-format`: read **paged Fixed Array** chunk indexes. A filtered,
fixed-dimension dataset with more than one data-block page (>1024 chunks by fixed-dimension dataset with more than one data-block page (>1024 chunks by
Binary file not shown.
@@ -695,6 +695,20 @@ fn v4_virtual_dataset_2d_same_file_read() {
); );
} }
#[test]
fn scaleoffset_float_escale_reads_as_raw() {
// The scale-offset filter's floating-point *E-scale* mode (cd_values[0] = 1)
// is not actually implemented by the HDF5 library: when asked for it, HDF5
// stores the chunk raw (no minbits/minval header) and sets the chunk filter
// mask to skip the filter. So such a dataset must read back verbatim purely
// by honoring the per-chunk filter mask — no E-scale decoder is needed.
let file_data = include_bytes!("fixtures/scaleoffset_float_escale.h5");
let (raw, datatype, _) = read_chunked_dataset(file_data, "x");
let values = read_as_f64(&raw, &datatype).unwrap();
let expect: Vec<f64> = (0..20).map(|i| i as f64 * 0.25).collect();
assert_eq!(values, expect, "E-scale (raw + masked filter) must read verbatim");
}
#[test] #[test]
fn v4_virtual_dataset_external_file_read() { fn v4_virtual_dataset_external_file_read() {
use clawhdf5_format::data_read::read_raw_data_full_with_resolver; use clawhdf5_format::data_read::read_raw_data_full_with_resolver;