fix(format): write paged files libhdf5 can open

FileWriter::with_page_size wrote a "version 4" superblock with an extra
page-size field. HDF5 has no superblock version 4, so libhdf5 refused
every such file ("bad superblock version number").

A paged file is now what libhdf5 itself writes for fs_strategy="page":
a v3 superblock whose extension object header holds a File Space Info
message (strategy PAGE, the page size, free space not persisted; same
bytes and flags as HDF5 2.0), with the file padded to a whole page.
h5py opens it, reports the strategy and page size, and can modify it in
r+ mode. Page sizes outside libhdf5's 512 B..1 GiB are an error.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-25 21:09:31 -05:00
co-authored by Claude Opus 5.5
parent be88e3fec7
commit 74fdf0582b
3 changed files with 163 additions and 21 deletions
+10 -3
View File
@@ -39,7 +39,13 @@ pub struct Superblock {
pub superblock_extension_address: Option<u64>,
/// CRC32C checksum (v2/v3 only).
pub checksum: Option<u32>,
/// Page size for page-buffer mode (v4 only). `None` for v0–v3.
/// Page size of the non-standard "version 4" superblock layout (v4 only).
/// `None` for v0–v3.
///
/// HDF5 has no superblock version 4 — libhdf5 refuses it. A real paged
/// file is a v2/v3 superblock whose extension holds a File Space Info
/// message (what `FileWriter::with_page_size` writes). This field is kept
/// only so such files written by older clawhdf5 versions still parse.
pub page_size: Option<u32>,
}
@@ -127,8 +133,9 @@ impl Superblock {
/// Serialize this superblock to bytes.
///
/// Writes v2/v3 format, or v4 (with `page_size`) when `self.version == 4`.
/// Computes and appends Jenkins lookup3 checksum.
/// Writes v2/v3 format, or the non-standard v4 (with `page_size`) when
/// `self.version == 4` — which no HDF5 library opens; see
/// [`Self::page_size`]. Computes and appends Jenkins lookup3 checksum.
pub fn serialize(&self) -> Vec<u8> {
let mut buf = Vec::with_capacity(48);
buf.extend_from_slice(&HDF5_SIGNATURE);