Commit Graph
3 Commits
Author SHA1 Message Date
osobhandClaude Opus 5.5 b6cbd2319f format: checked chunk addresses on the parallel read path
Three `chunk_info.address as usize` casts behind the `parallel` feature
survived the conversion, because check-32bit-casts.sh linted only default
features plus plugin-filters. On a 32-bit target with rayon a chunk address
past 4 GiB still wrapped onto another part of the file. They go through
addr::to_usize now, and the lane index (h % n, always < n) through
saturating_usize.

The script now lints no default features, default features, and every
optional feature but szip (wasm32; the set with zstd, which does not build
for wasm32, on the host, where the lint reports the same casts). With the
old parallel_read.rs/lane_partition.rs it fails listing the four casts; the
old script passed them. CHANGELOG and the design note give the exact count
(119) and what is not covered.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-26 14:08:57 -05:00
osobhandClaude Opus 5.5 1b4a93f65a docs: indexed name lookups and checked address conversion (M0)
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-26 13:33:24 -05:00
osobhandClaude Opus 5.5 1c1af460b6 docs: design for range reads (S3/HTTP, wasm lazy loading, SWMR, 32-bit)
Inventory of the 109 whole-file parser functions, a measurement of the
metadata ranges clawhdf5 touches on three corpus files against libhdf5,
options (storage trait, virtual slice, metadata prefetch, userfaultfd)
with how ros3, h5py+fsspec, pyfive, h5wasm and object_store do it, and
an incremental plan: M0 name-index lookups, M1 metadata over a Storage
trait, M2 raw data with batched ranges, M3 object_store backend,
M4 wasm fetch-driven lazy loading.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-26 10:17:02 -05:00