clawhdf5: FileEditor modifies existing files in place
New clawhdf5::FileEditor opens an HDF5 file (h5py-written at any libver, HDF5 2.0 format included, or clawhdf5-written) under an exclusive flock and changes only what an edit touches: - write_selection/write_all/write_values: compact, contiguous (also late-allocated) and chunked datasets, any selection. Chunks are decoded, updated and re-encoded; a filtered chunk that no longer fits moves to the end of the file unless it is the file's last structure, which grows in place. New chunks go into v1 B-tree, Extensible Array (paged data blocks included), Fixed Array and single-chunk indexes, created on first use. - resize: grow chunked datasets up to maxshape. - set_attr: add/replace compact attributes, in a NIL slot or a new continuation chunk. Each edit is planned in an in-memory image and refused whole (Error::Unsupported) when any part is unsupported (v2 B-tree / implicit new chunks, shrinking, vlen/reference data, dense or order-tracked attributes, cache images, paged/persistent free space). Commit writes and syncs new space before patching existing bytes. Layout v5 (HDF5 2.0) array indexes use 8-byte filtered chunk sizes, as libhdf5 does. Error gains Unsupported/InvalidArgument/Locked and is #[non_exhaustive]; the Python bindings map them. build_attr_message is public. Tests (h5py, h5dump, h5rs check --data after every round; h5py r+ afterwards): appends crossing EA super/data blocks and B-tree splits, the same B-tree node counts and EA statistics as libhdf5 for the same writes (in order, reversed and shuffled; paged blocks), every layout and chunk index overwritten under random selections, attributes to continuation chunks, random operations against a model, refused edits leave the file byte-identical, locking. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -66,7 +66,9 @@ fn _panic_for_test() -> PyResult<()> {
|
||||
/// - I/O errors -> `PyIOError`
|
||||
/// - Format/parsing errors -> `PyValueError`
|
||||
/// - Missing dataset/path errors -> `PyKeyError`
|
||||
/// - Other errors -> `PyOSError`
|
||||
/// - Invalid arguments -> `PyValueError`
|
||||
/// - Unsupported operations -> `PyNotImplementedError`
|
||||
/// - Other errors (a locked file, ...) -> `PyOSError`
|
||||
pub(crate) fn to_py_err(e: clawhdf5_rs::Error) -> PyErr {
|
||||
use clawhdf5_rs::Error;
|
||||
match &e {
|
||||
@@ -79,9 +81,14 @@ pub(crate) fn to_py_err(e: clawhdf5_rs::Error) -> PyErr {
|
||||
| Error::ZeroCopyNotContiguous
|
||||
| Error::ZeroCopyNonNativeEndian
|
||||
| Error::ZeroCopyTypeMismatch { .. }
|
||||
| Error::ZeroCopyUnaligned { .. } => {
|
||||
| Error::ZeroCopyUnaligned { .. }
|
||||
| Error::InvalidArgument(_) => {
|
||||
PyErr::new::<pyo3::exceptions::PyValueError, _>(e.to_string())
|
||||
}
|
||||
Error::Unsupported(_) => {
|
||||
PyErr::new::<pyo3::exceptions::PyNotImplementedError, _>(e.to_string())
|
||||
}
|
||||
_ => PyErr::new::<pyo3::exceptions::PyOSError, _>(e.to_string()),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user