fix(format): apply the base address of files with a user block
A file may start with a user block (h5py userblock_size, h5jam), putting the superblock at 512, 1024, ...; every address in the file is then relative to the superblock. The signature search found it, but every reader passed the whole file to the parsers, so addresses landed userblock bytes early and the root group failed with InvalidObjectHeaderVersion (twithub.h5, twithub513.h5, h5clear_fsm_persist_user_*.h5). Readers now view the file from the superblock on, taking the signature's position as the base address as libhdf5 does: File (mmap, buffered, from_bytes), MmapFile, LazyFile, AsyncHDF5File, the VOL and MPI VOL readers, the HNSW loader and external VDS source files. File, MmapFile and LazyFile gain user_block_size(). The new signature::split_user_block returns the two parts, and Superblock::parse refuses a non-zero offset (UserBlockNotStripped) so a format-level caller cannot silently apply superblock-relative addresses to the whole file. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -120,6 +120,11 @@ pub enum FormatError {
|
||||
/// A shared-message reference points at an object header that holds no
|
||||
/// (unshared) message of the referenced type (raw message type id).
|
||||
SharedMessageTargetMissing(u16),
|
||||
/// A superblock was parsed at a non-zero offset of the buffer (the file
|
||||
/// has a user block of this many bytes). HDF5 addresses are relative to
|
||||
/// the superblock, so the buffer must start there: see
|
||||
/// `signature::split_user_block`.
|
||||
UserBlockNotStripped(u64),
|
||||
/// A selection does not fit the dataset it was applied to (wrong rank, or
|
||||
/// it reaches past a dimension's extent).
|
||||
SelectionOutOfBounds(String),
|
||||
@@ -342,6 +347,11 @@ impl fmt::Display for FormatError {
|
||||
FormatError::SelectionOutOfBounds(msg) => {
|
||||
write!(f, "selection out of bounds: {msg}")
|
||||
}
|
||||
FormatError::UserBlockNotStripped(n) => write!(
|
||||
f,
|
||||
"file has a {n}-byte user block: parse the bytes from the superblock on \
|
||||
(signature::split_user_block)"
|
||||
),
|
||||
FormatError::SharedMessageTargetMissing(t) => write!(
|
||||
f,
|
||||
"shared message reference points at an object header with no message of type \
|
||||
|
||||
Reference in New Issue
Block a user