test: regression for scale-offset float E-scale (raw + masked filter)
The HDF5 library does not implement the scale-offset filter's floating-point E-scale mode. When asked for it (cd_values[0] = 1) it stores the chunk raw (no minbits/minval header) and sets the chunk filter mask to skip the filter, so such datasets read back verbatim purely by honoring the per-chunk filter mask — no E-scale decoder is required. Add a fixture written via the HDF5 low-level API (exact-representable values) and a test asserting it reads back verbatim, locking in the filter-mask path. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
@@ -39,6 +39,14 @@
|
|||||||
error. The previous `parse_vds_mappings` used a guessed layout that did not
|
error. The previous `parse_vds_mappings` used a guessed layout that did not
|
||||||
match real files and is replaced.
|
match real files and is replaced.
|
||||||
|
|
||||||
|
### Tests
|
||||||
|
- `clawhdf5-format`: regression test for **scale-offset float E-scale**
|
||||||
|
datasets. The HDF5 library does not implement E-scale encoding — when asked
|
||||||
|
for it (`cd_values[0] = 1`) it stores the chunk raw and sets the chunk filter
|
||||||
|
mask to skip the filter — so these files read back verbatim purely by
|
||||||
|
honoring the per-chunk filter mask. The test locks in that behavior against a
|
||||||
|
fixture produced via the HDF5 low-level API; no E-scale decoder is needed.
|
||||||
|
|
||||||
### Bug Fixes
|
### Bug Fixes
|
||||||
- `clawhdf5-format`: read **paged Fixed Array** chunk indexes. A filtered,
|
- `clawhdf5-format`: read **paged Fixed Array** chunk indexes. A filtered,
|
||||||
fixed-dimension dataset with more than one data-block page (>1024 chunks by
|
fixed-dimension dataset with more than one data-block page (>1024 chunks by
|
||||||
|
|||||||
Binary file not shown.
@@ -695,6 +695,20 @@ fn v4_virtual_dataset_2d_same_file_read() {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn scaleoffset_float_escale_reads_as_raw() {
|
||||||
|
// The scale-offset filter's floating-point *E-scale* mode (cd_values[0] = 1)
|
||||||
|
// is not actually implemented by the HDF5 library: when asked for it, HDF5
|
||||||
|
// stores the chunk raw (no minbits/minval header) and sets the chunk filter
|
||||||
|
// mask to skip the filter. So such a dataset must read back verbatim purely
|
||||||
|
// by honoring the per-chunk filter mask — no E-scale decoder is needed.
|
||||||
|
let file_data = include_bytes!("fixtures/scaleoffset_float_escale.h5");
|
||||||
|
let (raw, datatype, _) = read_chunked_dataset(file_data, "x");
|
||||||
|
let values = read_as_f64(&raw, &datatype).unwrap();
|
||||||
|
let expect: Vec<f64> = (0..20).map(|i| i as f64 * 0.25).collect();
|
||||||
|
assert_eq!(values, expect, "E-scale (raw + masked filter) must read verbatim");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn v4_virtual_dataset_external_file_read() {
|
fn v4_virtual_dataset_external_file_read() {
|
||||||
use clawhdf5_format::data_read::read_raw_data_full_with_resolver;
|
use clawhdf5_format::data_read::read_raw_data_full_with_resolver;
|
||||||
|
|||||||
Reference in New Issue
Block a user