format: checked chunk addresses on the parallel read path
Three `chunk_info.address as usize` casts behind the `parallel` feature survived the conversion, because check-32bit-casts.sh linted only default features plus plugin-filters. On a 32-bit target with rayon a chunk address past 4 GiB still wrapped onto another part of the file. They go through addr::to_usize now, and the lane index (h % n, always < n) through saturating_usize. The script now lints no default features, default features, and every optional feature but szip (wasm32; the set with zstd, which does not build for wasm32, on the host, where the lint reports the same casts). With the old parallel_read.rs/lane_partition.rs it fails listing the four casts; the old script passed them. CHANGELOG and the design note give the exact count (119) and what is not covered. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -5,8 +5,18 @@
|
||||
# of the file. File values go through `addr::to_usize` (a clean error) and
|
||||
# in-memory counts through `addr::saturating_usize`.
|
||||
#
|
||||
# Lints wasm32-unknown-unknown with clippy's cast_possible_truncation and
|
||||
# fails on any u64 -> usize finding (other truncations are not checked here).
|
||||
# Lints with clippy's cast_possible_truncation and fails on any u64 -> usize
|
||||
# finding (other truncations are not checked here), once per feature set
|
||||
# below. Together the sets compile every feature-gated line of the crate that
|
||||
# ci-test.sh builds: features only add code, except `not(feature = ...)`
|
||||
# paths for std/checksum/fast-checksum/szip, which the no-default-features
|
||||
# and default sets cover. szip is left out (it needs libaec), as in
|
||||
# ci-test.sh.
|
||||
#
|
||||
# The sets are linted for wasm32 where they build there. zstd links a C
|
||||
# library that does not build for wasm32, so the set with it is linted for
|
||||
# the host: the lint reports u64 -> usize casts whatever the target's
|
||||
# pointer width, and the crate has no pointer-width-dependent code.
|
||||
#
|
||||
# Usage:
|
||||
# ./scripts/check-32bit-casts.sh
|
||||
@@ -16,19 +26,42 @@
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
TARGET="wasm32-unknown-unknown"
|
||||
echo "==> Checking for truncating u64 -> usize casts in clawhdf5-format ($TARGET)"
|
||||
WASM="wasm32-unknown-unknown"
|
||||
ALL_BUT_ZSTD="parallel,lz4,pcodec,fast-checksum,blake3_hash,plugin-filters,lookup-stats"
|
||||
|
||||
out=$(cargo clippy -p clawhdf5-format --target "$TARGET" \
|
||||
--features plugin-filters --message-format short \
|
||||
-- -A clippy::all -W clippy::cast_possible_truncation 2>&1) || {
|
||||
echo "$out"
|
||||
echo "==> clippy failed" >&2
|
||||
exit 1
|
||||
}
|
||||
found=$(grep -F 'casting `u64` to `usize`' <<<"$out" || true)
|
||||
if [ -n "$found" ]; then
|
||||
echo "$found"
|
||||
# target|cargo feature arguments
|
||||
SETS=(
|
||||
"$WASM|--no-default-features"
|
||||
"$WASM|--no-default-features --features std,checksum"
|
||||
"$WASM|"
|
||||
"$WASM|--features $ALL_BUT_ZSTD"
|
||||
"host|--features $ALL_BUT_ZSTD,zstd"
|
||||
)
|
||||
|
||||
status=0
|
||||
for set in "${SETS[@]}"; do
|
||||
target=${set%%|*}
|
||||
args=${set#*|}
|
||||
target_args=()
|
||||
if [ "$target" != host ]; then
|
||||
target_args=(--target "$target")
|
||||
fi
|
||||
echo "==> Checking for truncating u64 -> usize casts in clawhdf5-format ($target: ${args:-default features})"
|
||||
# shellcheck disable=SC2086 # $args is a list of arguments
|
||||
out=$(cargo clippy -p clawhdf5-format "${target_args[@]}" $args \
|
||||
--message-format short \
|
||||
-- -A clippy::all -W clippy::cast_possible_truncation 2>&1) || {
|
||||
echo "$out"
|
||||
echo "==> clippy failed" >&2
|
||||
exit 1
|
||||
}
|
||||
found=$(grep -F 'casting `u64` to `usize`' <<<"$out" || true)
|
||||
if [ -n "$found" ]; then
|
||||
echo "$found"
|
||||
status=1
|
||||
fi
|
||||
done
|
||||
if [ "$status" -ne 0 ]; then
|
||||
echo "==> use addr::to_usize (file values) or addr::saturating_usize (in-memory counts)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
Reference in New Issue
Block a user