Omar SobhandClaude Sonnet 4.6
d6c4d4f111
All three SDD plans fully wired and committed to main: Filter Codecs (FC): - FC-1: Implement float E-scale in scaleoffset_decompress (value = minval + code * 2^E, negative exponents via cast to i32); add two round-trip tests. - FC-2: filters_szip.rs — feature-gated SZIP decode via libaec FFI; SZIP dispatch arm added to decompress_chunk. - FC-3: libaec-sys workspace crate with pkg-config probe and aec_buffer_decode FFI binding; added to workspace members. Format Write Extensions (FWE): - FWE-1: GroupBuilder::add_external_link() API; wired through FinishedGroup → GrpFlat → file_writer pass 1/2/3 (OH size, layout cursor, final write); external_link_write_roundtrip test. - FWE-2: data_layout_write.rs — serialize_vds_mappings with length_size param and version 0/1 (external vs same-file) selection; declared as pub mod. - FWE-3: with_virtual_sources empty-mapping guard (Important #9) — empty vec is silently ignored; vds_empty_mapping_list test updated to assert non-VDS layout results. MPI-IO VOL Backend (MPI): - MPI-1/2/3: mpi_vol.rs — MpiVol implementing VirtualObjectLayer; root-read + broadcast collective read; gather + root-write collective write; feature- gated mpi-io feature; wired into clawhdf5-io lib.rs. - MPI-4: mpi_io_bench binary (h5bench-equivalent MPI-IO throughput bench). mpi_vol.rs reviewer fixes: - Doc-comment updated to accurately describe root-read+broadcast pattern (not MPI_File_read_at); MpiVol::expected_capabilities() associated fn added so tests can verify capabilities without a live MPI universe; rank_and_size_stub_values renamed to no_feature_error_contains_feature_name. Workspace check: zero warnings, 20 test suites pass. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
103 lines
2.8 KiB
Rust
103 lines
2.8 KiB
Rust
//! Pure-Rust HDF5 binary format parsing and writing.
|
|
//!
|
|
//! `clawhdf5-format` is a zero-dependency (no C libhdf5) crate for reading and
|
|
//! writing HDF5 files. It supports `no_std` environments with the `alloc` crate.
|
|
//!
|
|
//! # Writing files
|
|
//!
|
|
//! Use [`file_writer::FileWriter`] to create HDF5 files:
|
|
//!
|
|
//! ```rust
|
|
//! use clawhdf5_format::file_writer::{FileWriter, AttrValue};
|
|
//!
|
|
//! let mut fw = FileWriter::new();
|
|
//! fw.create_dataset("data")
|
|
//! .with_f64_data(&[1.0, 2.0, 3.0])
|
|
//! .with_shape(&[3])
|
|
//! .set_attr("unit", AttrValue::String("m/s".into()));
|
|
//! let bytes = fw.finish().unwrap();
|
|
//! ```
|
|
//!
|
|
//! # Reading files
|
|
//!
|
|
//! Parsing follows the HDF5 object model: superblock → object header → messages.
|
|
//!
|
|
//! ```rust,no_run
|
|
//! use clawhdf5_format::{signature, superblock, object_header, group_v2,
|
|
//! datatype, dataspace, data_layout, data_read, message_type::MessageType};
|
|
//!
|
|
//! let file_data = std::fs::read("output.h5").unwrap();
|
|
//! let sig = signature::find_signature(&file_data).unwrap();
|
|
//! let sb = superblock::Superblock::parse(&file_data, sig).unwrap();
|
|
//! let addr = group_v2::resolve_path_any(&file_data, &sb, "data").unwrap();
|
|
//! let hdr = object_header::ObjectHeader::parse(
|
|
//! &file_data, addr as usize, sb.offset_size, sb.length_size).unwrap();
|
|
//! ```
|
|
//!
|
|
//! # Features
|
|
//!
|
|
//! | Feature | Default | Description |
|
|
//! |---------|---------|-------------|
|
|
//! | `std` | yes | Standard library support |
|
|
//! | `checksum` | yes | Jenkins lookup3 checksum validation |
|
|
//! | `deflate` | yes | Deflate (gzip) compression via `flate2` |
|
|
//! | `provenance` | yes | SHINES provenance — SHA-256 hashing & verification |
|
|
|
|
#![cfg_attr(not(feature = "std"), no_std)]
|
|
|
|
#[cfg(not(feature = "std"))]
|
|
extern crate alloc;
|
|
|
|
pub mod attribute;
|
|
pub mod attribute_info;
|
|
pub mod btree_v1;
|
|
pub mod btree_v2;
|
|
pub mod checksum;
|
|
pub mod chunk_cache;
|
|
pub mod chunk_index;
|
|
pub mod chunked_read;
|
|
pub mod chunked_write;
|
|
pub mod data_layout;
|
|
pub mod data_layout_write;
|
|
pub mod data_read;
|
|
pub mod dataspace;
|
|
pub mod datatype;
|
|
pub mod dict_encoding;
|
|
pub mod ea_writer;
|
|
pub mod error;
|
|
pub mod extensible_array;
|
|
pub mod file_writer;
|
|
pub mod filter_pipeline;
|
|
pub mod filters;
|
|
mod filters_szip;
|
|
pub mod fixed_array;
|
|
pub mod fractal_heap;
|
|
pub mod global_heap;
|
|
pub mod group_info;
|
|
pub mod group_v1;
|
|
pub mod group_v2;
|
|
#[cfg(feature = "parallel")]
|
|
pub mod lane_partition;
|
|
pub mod link_info;
|
|
pub mod link_message;
|
|
pub mod local_heap;
|
|
pub mod message_type;
|
|
pub mod metadata_cache;
|
|
pub mod metadata_index;
|
|
pub mod object_header;
|
|
pub mod object_header_writer;
|
|
#[cfg(feature = "parallel")]
|
|
pub mod parallel_read;
|
|
pub mod profiling;
|
|
pub mod property_list;
|
|
pub mod selection;
|
|
pub mod shared_message;
|
|
pub mod signature;
|
|
pub mod superblock;
|
|
pub mod symbol_table;
|
|
pub mod type_builders;
|
|
pub mod vl_data;
|
|
|
|
#[cfg(feature = "provenance")]
|
|
pub mod provenance;
|