perf: copy contiguous hyperslab and point reads run by run

A 256 x 256 hyperslab of a contiguous f32 dataset read at an eighth of
h5py's speed: partial_read copied the bounding box out of the file, the
extractor then walked it element by element (a recursive call and two
bounds checks per element) into a second buffer, and read_f32_selection
converted that into a third.

Selections of contiguous data are now copied straight from the file, one
memcpy per run of elements contiguous in the file (gather.rs: a block
along the last dimension, touching blocks as one range, whole rows
merged), with no zero-filled intermediate and no full copy for large
selections. The typed selection readers copy into their Vec<T> directly
when the dataset stores T natively (new data_read::read_selection_native
and sealed NativeElement trait, which the read_as_* fast paths now share;
read_as_u64 gains one) and convert as before otherwise. The general
extractor used by the chunked paths runs on the same run walker, keeping
its old handling of unvalidated selections.

Checked against h5py (contiguous_read_interop.rs) for strided, blocked,
adjacent-block and whole-row hyperslabs, points and empty selections of
every 1-8-byte type in both byte orders, ranks 1-4.

Also keeps the huge-page threshold constant out of no_std builds, where
it was unused.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 08:21:47 -05:00
co-authored by Claude Opus 5.5
parent 78c769f179
commit 2bc4cb46a6
8 changed files with 586 additions and 170 deletions
+1
View File
@@ -94,6 +94,7 @@ mod filters_szip;
pub mod fixed_array;
pub mod float16;
pub mod fractal_heap;
mod gather;
pub mod global_heap;
pub mod group_info;
pub mod group_v1;