osobhandClaude Opus 5.5
42894bf93b
BTreeV2Header::parse_in, collect_btree_v2_records_in and find_btree_v2_records_in read one bounded window per node (its size is known from the parent before the node is read; a count stretched past node_size is checked against the end of the file first), with the whole-file bounds errors unchanged. With them, dense attributes, a SOHM B-tree index and huge fractal-heap objects no longer answer ContiguousStorageRequired, and group_v1/group_v2 listings, lookups and path resolution get *_in cores (resolve_group_children_in, resolve_child_in, resolve_path_any_in, ...). The &[u8] functions are thin wrappers, as in M1. The equivalence harness now fails on any ContiguousStorageRequired and compares v2 B-tree headers, records and descents, group listings, child lookups and paths; a unit test compares a two-level tree through a read_at-only storage truncated at every length and with every node byte flipped. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
122 lines
4.7 KiB
Rust
122 lines
4.7 KiB
Rust
//! File address and length → in-memory index conversion.
|
|
//!
|
|
//! HDF5 addresses and lengths are 64-bit; the file is parsed through a
|
|
//! `&[u8]` indexed by `usize`. On a 64-bit target every `u64` fits, but on a
|
|
//! 32-bit one (`wasm32`, `i686`, `thumbv7em`) an address past `usize::MAX`
|
|
//! used to be truncated by an `as usize` cast — silently pointing at another
|
|
//! part of the file — or to panic. [`to_usize`] is the one conversion the
|
|
//! parsers use instead: such an address is a clean
|
|
//! [`FormatError::Overflow`]. It cannot be inside the data anyway: no slice
|
|
//! is longer than `isize::MAX` bytes.
|
|
|
|
#[cfg(not(feature = "std"))]
|
|
use alloc::format;
|
|
|
|
use crate::error::FormatError;
|
|
|
|
/// A file address, offset or length from the file as a `usize` index.
|
|
///
|
|
/// Fails with [`FormatError::Overflow`] when the value does not fit this
|
|
/// platform's `usize` (only possible on targets narrower than 64 bits).
|
|
#[inline]
|
|
pub fn to_usize(value: u64) -> Result<usize, FormatError> {
|
|
to_index::<usize>(value)
|
|
}
|
|
|
|
/// A file address for a [`crate::storage::Storage`] read, checked as
|
|
/// [`to_usize`] checks it: the parsers read through 64-bit offsets, but an
|
|
/// address that could not index an in-memory file on this platform is the
|
|
/// same [`FormatError::Overflow`] the slice parsers gave for it.
|
|
#[inline]
|
|
pub fn checked_addr(value: u64) -> Result<u64, FormatError> {
|
|
to_usize(value).map(|_| value)
|
|
}
|
|
|
|
/// [`to_usize`] for an index type of any width. `usize` is 64 bits wide on
|
|
/// the hosts CI tests on, where the error path cannot be reached through
|
|
/// `usize`; tests run the same code with `u32` in its place, as on a 32-bit
|
|
/// target.
|
|
#[inline]
|
|
fn to_index<T: TryFrom<u64>>(value: u64) -> Result<T, FormatError> {
|
|
T::try_from(value).map_err(|_| too_large(value))
|
|
}
|
|
|
|
/// A count or offset into an in-memory buffer (a codec's progress counter,
|
|
/// a size the writer computed from data it holds) as a `usize`, saturating
|
|
/// at `usize::MAX` instead of truncating.
|
|
///
|
|
/// For values that are bounded by the length of something in memory, so
|
|
/// always fit; if one ever did not, a saturated index fails its bounds check
|
|
/// or allocation instead of silently addressing the wrong bytes. A value
|
|
/// read from the file uses [`to_usize`].
|
|
#[inline]
|
|
pub fn saturating_usize(value: u64) -> usize {
|
|
saturating_index(value, usize::MAX)
|
|
}
|
|
|
|
/// [`saturating_usize`] for an index type of any width, whose largest
|
|
/// value is `max` (see [`to_index`]).
|
|
#[inline]
|
|
fn saturating_index<T: TryFrom<u64>>(value: u64, max: T) -> T {
|
|
T::try_from(value).unwrap_or(max)
|
|
}
|
|
|
|
#[cold]
|
|
#[inline(never)]
|
|
fn too_large(value: u64) -> FormatError {
|
|
FormatError::Overflow(format!(
|
|
"file address or length {value:#x} exceeds this platform's address space"
|
|
))
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn values_that_fit_convert_exactly() {
|
|
assert_eq!(to_usize(0), Ok(0));
|
|
assert_eq!(to_usize(0x1234), Ok(0x1234));
|
|
assert_eq!(to_usize(usize::MAX as u64), Ok(usize::MAX));
|
|
}
|
|
|
|
#[test]
|
|
fn saturating_conversion_never_wraps() {
|
|
assert_eq!(saturating_usize(0), 0);
|
|
assert_eq!(saturating_usize(0x1234), 0x1234);
|
|
assert_eq!(saturating_usize(usize::MAX as u64), usize::MAX);
|
|
// Past usize::MAX (32-bit targets) or at u64::MAX: saturates.
|
|
assert_eq!(saturating_usize(u64::MAX), usize::MAX);
|
|
}
|
|
|
|
#[test]
|
|
fn values_past_usize_max_are_an_error_not_truncated() {
|
|
// Reachable through `usize` only where it is narrower than u64 (no
|
|
// such target runs tests in CI), so the same conversion is run with
|
|
// u32 standing in for a 32-bit usize.
|
|
let max = u64::from(u32::MAX);
|
|
assert_eq!(to_index::<u32>(max), Ok(u32::MAX));
|
|
for past in [max + 1, max + 0x10, 0x1_0000_1234, u64::MAX] {
|
|
let err = to_index::<u32>(past).unwrap_err();
|
|
assert!(
|
|
matches!(err, FormatError::Overflow(_)),
|
|
"{past:#x}: {err:?}"
|
|
);
|
|
}
|
|
// Where an `as` cast would have wrapped to a small, valid-looking
|
|
// index, it is not returned.
|
|
assert_eq!(0x1_0000_1234_u64 as u32, 0x1234);
|
|
assert!(to_index::<u32>(0x1_0000_1234).is_err());
|
|
|
|
assert_eq!(saturating_index(max + 1, u32::MAX), u32::MAX);
|
|
assert_eq!(saturating_index(0x1_0000_1234, u32::MAX), u32::MAX);
|
|
assert_eq!(saturating_index(0x1234, u32::MAX), 0x1234);
|
|
|
|
// And through `usize` itself, whichever width it has here.
|
|
match (usize::MAX as u64).checked_add(1) {
|
|
Some(past) => assert!(matches!(to_usize(past), Err(FormatError::Overflow(_)))),
|
|
None => assert_eq!(to_usize(u64::MAX), Ok(usize::MAX)),
|
|
}
|
|
}
|
|
}
|