quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases 1 Wiki Activity

format: v2 B-trees, dense groups and group listings over Storage

Browse Source 
BTreeV2Header::parse_in, collect_btree_v2_records_in and
find_btree_v2_records_in read one bounded window per node (its size is
known from the parent before the node is read; a count stretched past
node_size is checked against the end of the file first), with the
whole-file bounds errors unchanged. With them, dense attributes, a SOHM
B-tree index and huge fractal-heap objects no longer answer
ContiguousStorageRequired, and group_v1/group_v2 listings, lookups and
path resolution get *_in cores (resolve_group_children_in,
resolve_child_in, resolve_path_any_in, ...). The &[u8] functions are
thin wrappers, as in M1.

The equivalence harness now fails on any ContiguousStorageRequired and
compares v2 B-tree headers, records and descents, group listings, child
lookups and paths; a unit test compares a two-level tree through a
read_at-only storage truncated at every length and with every node byte
flipped.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 16:13:01 -05:00
co-authored by Claude Opus 5.5
parent 8f59b2e1c2
commit 42894bf93b
9 changed files with 544 additions and 297 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/clawhdf5-format/src/shared_message.rs
+10 -13
View File
@@ -23,12 +23,12 @@ use alloc::vec::Vec;
#[cfg(feature = "std")]
use std::borrow::Cow;
use crate::btree_v2::{BTreeV2Header, collect_btree_v2_records};
use crate::btree_v2::{BTreeV2Header, collect_btree_v2_records_in};
use crate::error::FormatError;
use crate::fractal_heap::FractalHeapHeader;
use crate::message_type::MessageType;
use crate::object_header::ObjectHeader;
use crate::storage::{Storage, Window, read_exact_at, require_contiguous};
use crate::storage::{Storage, Window, read_exact_at};
/// Fractal heap ID length for SOHM entries (fixed at 8 bytes).
const FHEAP_ID_LEN: usize = 8;
@@ -423,19 +423,15 @@ pub fn parse_sohm_btree_entries(
parse_sohm_btree_entries_in(file_data, btree_addr as u64, offset_size, length_size)
}
/// [`parse_sohm_btree_entries`] over any [`Storage`]. The v2 B-tree is not
/// read over [`Storage`] yet, so this needs the whole file in memory
/// ([`FormatError::ContiguousStorageRequired`] otherwise).
/// [`parse_sohm_btree_entries`] over any [`Storage`].
pub fn parse_sohm_btree_entries_in<S: Storage + ?Sized>(
file: &S,
btree_addr: u64,
offset_size: u8,
length_size: u8,
) -> Result<Vec<SohmEntry>, FormatError> {
let file_data = require_contiguous(file, "a shared-message B-tree index")?;
let btree_addr = usize::try_from(btree_addr).unwrap_or(usize::MAX);
let header = BTreeV2Header::parse(file_data, btree_addr, offset_size, length_size)?;
let records = collect_btree_v2_records(file_data, &header, offset_size, length_size)?;
let header = BTreeV2Header::parse_in(file, btree_addr, offset_size, length_size)?;
let records = collect_btree_v2_records_in(file, &header, offset_size, length_size)?;
let mut entries = Vec::with_capacity(records.len());
for rec in &records {
let entry = parse_sohm_entry(&rec.data, offset_size)?;
@@ -1243,11 +1239,12 @@ mod tests {
}
}
assert!(compared > 200);
// The B-tree index is not read over Storage yet: a clean error.
let st = CountingStorage::new(vec![0u8; 64]);
// The B-tree index reads through Storage too, errors included.
let junk = vec![0u8; 64];
let st = CountingStorage::new(junk.clone());
assert_eq!(
parse_sohm_btree_entries_in(&st, 0, 8, 8).unwrap_err(),
FormatError::ContiguousStorageRequired("a shared-message B-tree index")
format!("{:?}", parse_sohm_btree_entries_in(&st, 0, 8, 8)),
format!("{:?}", parse_sohm_btree_entries(&junk, 0, 8, 8))
);
}
}