quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

format: checked chunk addresses on the parallel read path

Browse Source 
Three `chunk_info.address as usize` casts behind the `parallel` feature
survived the conversion, because check-32bit-casts.sh linted only default
features plus plugin-filters. On a 32-bit target with rayon a chunk address
past 4 GiB still wrapped onto another part of the file. They go through
addr::to_usize now, and the lane index (h % n, always < n) through
saturating_usize.

The script now lints no default features, default features, and every
optional feature but szip (wasm32; the set with zstd, which does not build
for wasm32, on the host, where the lint reports the same casts). With the
old parallel_read.rs/lane_partition.rs it fails listing the four casts; the
old script passed them. CHANGELOG and the design note give the exact count
(119) and what is not covered.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 14:08:57 -05:00
co-authored by Claude Opus 5.5
parent 92c8285549
commit b6cbd2319f
5 changed files with 68 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+10 -3
View File
@@ -28,14 +28,21 @@
resolves it in dense and compact groups.
### Checked address conversion (2026-09-26)
- **No 64-bit file value is truncated on a 32-bit target.** Every
`u64 as usize` cast in `clawhdf5-format` (115) is gone: file addresses,
- **No 64-bit file value is truncated on a 32-bit target.** All 119
truncating `u64 as usize` casts in `clawhdf5-format` that clippy's
`cast_possible_truncation` reports, under every feature the crate is
built with in CI except `szip` (115 with default features and
`plugin-filters`, 4 more behind `parallel`), are gone: file addresses,
lengths and counts go through `addr::to_usize`, which fails with
`FormatError::Overflow` where the value does not fit (wasm32 and other
32-bit targets; it used to wrap onto another part of the file), and
in-memory counts through `addr::saturating_usize`. On 64-bit targets
nothing changes. `scripts/check-32bit-casts.sh` (run by `ci-test.sh`)
lints the wasm32 build and fails on any new truncating cast.
lints the crate with no default features, with default features, and
with every optional feature but `szip` (for wasm32; the set with `zstd`,
which does not build for wasm32, for the host), and fails on any new
truncating cast. The facade, `clawhdf5-io` and `clawhdf5-ann` are not
covered.
### Chunked full reads (2026-09-26)
- **Chunks are decoded straight into the output, into reused buffers.** A