osobhandClaude Opus 5.5
bba1560416
Both indexes place each chunk at a linear index computed from the dataset's maximum dimensions (libhdf5's max_down_chunks), and the Extensible Array first swizzles its unlimited dimension to the slowest position. We linearised by the current dimensions, so any dataset whose shape was smaller than its maxshape, or whose unlimited dimension was not the first, read back scrambled without an error: h5py libver="latest" files with maxshape (10, None) or (20, 10), and the libhdf5 test files h5fc_ext*.h5 and test_ld.h5. The linearisation now lives in chunk_grid (shared with the writers), and slots beyond the current extent are ignored as the library does. read_fixed_array_chunks / read_extensible_array_chunks take the dataspace's max dimensions. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
107 lines
2.9 KiB
Rust
107 lines
2.9 KiB
Rust
//! Pure-Rust HDF5 binary format parsing and writing.
|
|
//!
|
|
//! `clawhdf5-format` is a zero-dependency (no C libhdf5) crate for reading and
|
|
//! writing HDF5 files. It supports `no_std` environments with the `alloc` crate.
|
|
//!
|
|
//! # Writing files
|
|
//!
|
|
//! Use [`file_writer::FileWriter`] to create HDF5 files:
|
|
//!
|
|
//! ```rust
|
|
//! use clawhdf5_format::file_writer::{FileWriter, AttrValue};
|
|
//!
|
|
//! let mut fw = FileWriter::new();
|
|
//! fw.create_dataset("data")
|
|
//! .with_f64_data(&[1.0, 2.0, 3.0])
|
|
//! .with_shape(&[3])
|
|
//! .set_attr("unit", AttrValue::String("m/s".into()));
|
|
//! let bytes = fw.finish().unwrap();
|
|
//! ```
|
|
//!
|
|
//! # Reading files
|
|
//!
|
|
//! Parsing follows the HDF5 object model: superblock → object header → messages.
|
|
//!
|
|
//! ```rust,no_run
|
|
//! use clawhdf5_format::{signature, superblock, object_header, group_v2,
|
|
//! datatype, dataspace, data_layout, data_read, message_type::MessageType};
|
|
//!
|
|
//! let file_data = std::fs::read("output.h5").unwrap();
|
|
//! let sig = signature::find_signature(&file_data).unwrap();
|
|
//! let sb = superblock::Superblock::parse(&file_data, sig).unwrap();
|
|
//! let addr = group_v2::resolve_path_any(&file_data, &sb, "data").unwrap();
|
|
//! let hdr = object_header::ObjectHeader::parse(
|
|
//! &file_data, addr as usize, sb.offset_size, sb.length_size).unwrap();
|
|
//! ```
|
|
//!
|
|
//! # Features
|
|
//!
|
|
//! | Feature | Default | Description |
|
|
//! |---------|---------|-------------|
|
|
//! | `std` | yes | Standard library support |
|
|
//! | `checksum` | yes | Jenkins lookup3 checksum validation |
|
|
//! | `deflate` | yes | Deflate (gzip) compression via `flate2` |
|
|
//! | `provenance` | yes | SHINES provenance — SHA-256 hashing & verification |
|
|
|
|
#![cfg_attr(not(feature = "std"), no_std)]
|
|
|
|
#[cfg(not(feature = "std"))]
|
|
extern crate alloc;
|
|
|
|
pub mod attribute;
|
|
pub mod attribute_info;
|
|
pub mod btree_v1;
|
|
pub mod btree_v2;
|
|
pub mod checksum;
|
|
pub mod chunk_cache;
|
|
mod chunk_grid;
|
|
pub mod chunk_index;
|
|
pub mod chunked_read;
|
|
pub mod chunked_write;
|
|
pub mod data_layout;
|
|
pub mod data_layout_write;
|
|
pub mod data_read;
|
|
pub mod dataspace;
|
|
pub mod datatype;
|
|
pub mod dict_encoding;
|
|
pub mod ea_writer;
|
|
pub mod error;
|
|
pub mod extensible_array;
|
|
pub mod file_writer;
|
|
pub mod fill_value;
|
|
pub mod filter_pipeline;
|
|
pub mod filters;
|
|
mod filters_szip;
|
|
pub mod fixed_array;
|
|
pub mod float16;
|
|
pub mod fractal_heap;
|
|
pub mod global_heap;
|
|
pub mod group_info;
|
|
pub mod group_v1;
|
|
pub mod group_v2;
|
|
#[cfg(feature = "parallel")]
|
|
pub mod lane_partition;
|
|
pub mod link_info;
|
|
pub mod link_message;
|
|
pub mod local_heap;
|
|
pub mod message_type;
|
|
pub mod metadata_cache;
|
|
pub mod metadata_index;
|
|
pub mod object_header;
|
|
pub mod object_header_writer;
|
|
#[cfg(feature = "parallel")]
|
|
pub mod parallel_read;
|
|
pub mod partial_read;
|
|
pub mod profiling;
|
|
pub mod property_list;
|
|
pub mod selection;
|
|
pub mod shared_message;
|
|
pub mod signature;
|
|
pub mod superblock;
|
|
pub mod symbol_table;
|
|
pub mod type_builders;
|
|
pub mod vl_data;
|
|
|
|
#[cfg(feature = "provenance")]
|
|
pub mod provenance;
|