fix(format): apply the base address of files with a user block
A file may start with a user block (h5py userblock_size, h5jam), putting the superblock at 512, 1024, ...; every address in the file is then relative to the superblock. The signature search found it, but every reader passed the whole file to the parsers, so addresses landed userblock bytes early and the root group failed with InvalidObjectHeaderVersion (twithub.h5, twithub513.h5, h5clear_fsm_persist_user_*.h5). Readers now view the file from the superblock on, taking the signature's position as the base address as libhdf5 does: File (mmap, buffered, from_bytes), MmapFile, LazyFile, AsyncHDF5File, the VOL and MPI VOL readers, the HNSW loader and external VDS source files. File, MmapFile and LazyFile gain user_block_size(). The new signature::split_user_block returns the two parts, and Superblock::parse refuses a non-zero offset (UserBlockNotStripped) so a format-level caller cannot silently apply superblock-relative addresses to the whole file. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -174,8 +174,18 @@ impl Superblock {
|
||||
|
||||
/// Parse a superblock from `data` starting at `signature_offset`.
|
||||
///
|
||||
/// The signature must be present at the given offset.
|
||||
/// The signature must be present at the given offset, and that offset
|
||||
/// must be 0: every address in an HDF5 file is relative to the
|
||||
/// superblock, so when a file has a user block (signature at 512, 1024,
|
||||
/// …) the caller must pass the bytes from the signature on — see
|
||||
/// [`crate::signature::split_user_block`] — and use that slice as
|
||||
/// `file_data` everywhere. A non-zero offset is refused with
|
||||
/// [`FormatError::UserBlockNotStripped`] because the addresses in the
|
||||
/// returned superblock would otherwise be applied to the wrong bytes.
|
||||
pub fn parse(data: &[u8], signature_offset: usize) -> Result<Superblock, FormatError> {
|
||||
if signature_offset != 0 {
|
||||
return Err(FormatError::UserBlockNotStripped(signature_offset as u64));
|
||||
}
|
||||
let d = data
|
||||
.get(signature_offset..)
|
||||
.ok_or(FormatError::UnexpectedEof {
|
||||
@@ -676,7 +686,16 @@ mod tests {
|
||||
let mut data = vec![0u8; 1024];
|
||||
let v0 = build_v0_bytes(8);
|
||||
data[512..512 + v0.len()].copy_from_slice(&v0);
|
||||
let sb = Superblock::parse(&data, 512).unwrap();
|
||||
// Addresses are relative to the superblock, so parsing in place
|
||||
// (where they would be applied to the whole buffer) is refused...
|
||||
assert_eq!(
|
||||
Superblock::parse(&data, 512),
|
||||
Err(FormatError::UserBlockNotStripped(512))
|
||||
);
|
||||
// ...and the caller parses the bytes from the signature on.
|
||||
let (ub, hdf5) = crate::signature::split_user_block(&data).unwrap();
|
||||
assert_eq!(ub.len(), 512);
|
||||
let sb = Superblock::parse(hdf5, 0).unwrap();
|
||||
assert_eq!(sb.version, 0);
|
||||
assert_eq!(sb.root_group_address, 96);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user