feat: implement filter codecs, format write extensions, and MPI-IO VOL

All three SDD plans fully wired and committed to main:

Filter Codecs (FC):
- FC-1: Implement float E-scale in scaleoffset_decompress (value = minval +
  code * 2^E, negative exponents via cast to i32); add two round-trip tests.
- FC-2: filters_szip.rs — feature-gated SZIP decode via libaec FFI; SZIP
  dispatch arm added to decompress_chunk.
- FC-3: libaec-sys workspace crate with pkg-config probe and aec_buffer_decode
  FFI binding; added to workspace members.

Format Write Extensions (FWE):
- FWE-1: GroupBuilder::add_external_link() API; wired through FinishedGroup
  → GrpFlat → file_writer pass 1/2/3 (OH size, layout cursor, final write);
  external_link_write_roundtrip test.
- FWE-2: data_layout_write.rs — serialize_vds_mappings with length_size param
  and version 0/1 (external vs same-file) selection; declared as pub mod.
- FWE-3: with_virtual_sources empty-mapping guard (Important #9) — empty vec
  is silently ignored; vds_empty_mapping_list test updated to assert non-VDS
  layout results.

MPI-IO VOL Backend (MPI):
- MPI-1/2/3: mpi_vol.rs — MpiVol implementing VirtualObjectLayer; root-read
  + broadcast collective read; gather + root-write collective write; feature-
  gated mpi-io feature; wired into clawhdf5-io lib.rs.
- MPI-4: mpi_io_bench binary (h5bench-equivalent MPI-IO throughput bench).

mpi_vol.rs reviewer fixes:
- Doc-comment updated to accurately describe root-read+broadcast pattern
  (not MPI_File_read_at); MpiVol::expected_capabilities() associated fn
  added so tests can verify capabilities without a live MPI universe;
  rank_and_size_stub_values renamed to no_feature_error_contains_feature_name.

Workspace check: zero warnings, 20 test suites pass.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
This commit is contained in:
Omar Sobh
2026-06-30 11:41:41 +00:00
co-authored by Claude Sonnet 4.6
parent 90bdd7cd13
commit d6c4d4f111
16 changed files with 1121 additions and 41 deletions
+36
View File
@@ -0,0 +1,36 @@
//! Raw FFI bindings to libaec (Adaptive Entropy Coding library).
//!
//! Provides the `aec_buffer_decode` convenience function for one-shot decompression.
// AEC flag constants matching aec.h
pub const AEC_DATA_PREPROCESS: u32 = 1; // NN preprocessing
pub const AEC_DATA_MSB: u32 = 2; // big-endian sample order
pub const AEC_RESTRICTED: u32 = 4; // restricted coding set
pub const AEC_ALLOW_K13: u32 = 8; // allow k=13 option
unsafe extern "C" {
/// One-shot decompression. Returns 0 on success.
///
/// # Safety
/// `src` must be valid for `src_len` bytes; `dst` must be valid for `*dst_len` bytes.
pub fn aec_buffer_decode(
src: *const u8,
src_len: usize,
dst: *mut u8,
dst_len: *mut usize,
bits_per_sample: u32,
block_size: u32,
flags: u32,
) -> i32;
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn constants_are_correct() {
assert_eq!(AEC_DATA_PREPROCESS, 1);
assert_eq!(AEC_DATA_MSB, 2);
}
}