clawhdf5: FileEditor modifies existing files in place

New clawhdf5::FileEditor opens an HDF5 file (h5py-written at any libver,
HDF5 2.0 format included, or clawhdf5-written) under an exclusive flock
and changes only what an edit touches:
- write_selection/write_all/write_values: compact, contiguous (also
  late-allocated) and chunked datasets, any selection. Chunks are decoded,
  updated and re-encoded; a filtered chunk that no longer fits moves to the
  end of the file unless it is the file's last structure, which grows in
  place. New chunks go into v1 B-tree, Extensible Array (paged data blocks
  included), Fixed Array and single-chunk indexes, created on first use.
- resize: grow chunked datasets up to maxshape.
- set_attr: add/replace compact attributes, in a NIL slot or a new
  continuation chunk.
Each edit is planned in an in-memory image and refused whole
(Error::Unsupported) when any part is unsupported (v2 B-tree / implicit
new chunks, shrinking, vlen/reference data, dense or order-tracked
attributes, cache images, paged/persistent free space). Commit writes and
syncs new space before patching existing bytes. Layout v5 (HDF5 2.0)
array indexes use 8-byte filtered chunk sizes, as libhdf5 does.

Error gains Unsupported/InvalidArgument/Locked and is #[non_exhaustive];
the Python bindings map them. build_attr_message is public.

Tests (h5py, h5dump, h5rs check --data after every round; h5py r+
afterwards): appends crossing EA super/data blocks and B-tree splits, the
same B-tree node counts and EA statistics as libhdf5 for the same writes
(in order, reversed and shuffled; paged blocks), every layout and chunk
index overwritten under random selections, attributes to continuation
chunks, random operations against a model, refused edits leave the file
byte-identical, locking.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 13:35:06 -05:00
co-authored by Claude Opus 5.5
parent f2ff2c424f
commit 3c89a31df0
14 changed files with 4657 additions and 7 deletions
+9 -2
View File
@@ -66,7 +66,9 @@ fn _panic_for_test() -> PyResult<()> {
/// - I/O errors -> `PyIOError`
/// - Format/parsing errors -> `PyValueError`
/// - Missing dataset/path errors -> `PyKeyError`
/// - Other errors -> `PyOSError`
/// - Invalid arguments -> `PyValueError`
/// - Unsupported operations -> `PyNotImplementedError`
/// - Other errors (a locked file, ...) -> `PyOSError`
pub(crate) fn to_py_err(e: clawhdf5_rs::Error) -> PyErr {
use clawhdf5_rs::Error;
match &e {
@@ -79,9 +81,14 @@ pub(crate) fn to_py_err(e: clawhdf5_rs::Error) -> PyErr {
| Error::ZeroCopyNotContiguous
| Error::ZeroCopyNonNativeEndian
| Error::ZeroCopyTypeMismatch { .. }
| Error::ZeroCopyUnaligned { .. } => {
| Error::ZeroCopyUnaligned { .. }
| Error::InvalidArgument(_) => {
PyErr::new::<pyo3::exceptions::PyValueError, _>(e.to_string())
}
Error::Unsupported(_) => {
PyErr::new::<pyo3::exceptions::PyNotImplementedError, _>(e.to_string())
}
_ => PyErr::new::<pyo3::exceptions::PyOSError, _>(e.to_string()),
}
}