osobhandClaude Fable 5.1
926dc457e0
Compound datasets written with default libver bounds (datatype message version 1, i.e. plain h5py.File(path, 'w')) could not be read: the v1 member layout has 28 bytes of legacy array fields after the byte offset (dimensionality 1, reserved 3, permutation 4, reserved 4, four sizes 16) and the parser skipped 24, so every following member was read 4 bytes off. v2 was also wrong: it keeps the 8-byte name padding and has no array fields. Found by adding a default-libver axis to the h5py-generated-file tests (HDF5 2.0 raised the default low bound to 1.8, so "default" files are a distinct format path from libver='latest'). Adds byte-level v1/v2 regression tests, a truncation test, and fuzz corpus seeds for v1 compound and native complex. Co-Authored-By: Claude Fable 5.1 <[email protected]>
Fuzz Testing for clawhdf5-format
Uses cargo-fuzz (libFuzzer) to test parser robustness against malformed inputs.
Prerequisites
cargo install cargo-fuzz
rustup toolchain install nightly
Fuzz Targets
| Target | Parser | Description |
|---|---|---|
fuzz_superblock |
Superblock::parse |
Superblock parsing (v0-v3) with signature search |
fuzz_object_header |
ObjectHeader::parse |
Object header v1/v2 with various offset/length sizes |
fuzz_datatype |
Datatype::parse |
All 12 HDF5 datatype classes (recursive) |
fuzz_dataspace |
Dataspace::parse |
Dataspace messages with various length sizes |
fuzz_fractal_heap |
FractalHeapHeader::parse |
Fractal heap header parsing |
fuzz_btree_v2 |
BTreeV2Header::parse |
B-tree v2 header parsing |
fuzz_filter_pipeline |
FilterPipeline::parse |
Filter pipeline messages (v1/v2) |
fuzz_full_file |
signature + superblock + root group | End-to-end file parsing chain |
fuzz_dataset_read |
Dataset::read_* (via clawhdf5) |
Walks every dataset in the parsed file and exercises the contiguous/chunked/compact raw-data read paths (chunked_read.rs, data_read.rs) that fuzz_full_file doesn't reach |
Running
Run a single target (runs indefinitely until stopped or a crash is found):
cd crates/clawhdf5-format
cargo +nightly fuzz run fuzz_datatype
Run with a time limit (seconds):
cargo +nightly fuzz run fuzz_datatype -- -max_total_time=60
Run all targets for 30 seconds each:
for target in fuzz_superblock fuzz_object_header fuzz_datatype fuzz_dataspace \
fuzz_fractal_heap fuzz_btree_v2 fuzz_filter_pipeline fuzz_full_file \
fuzz_dataset_read; do
echo "=== $target ==="
cargo +nightly fuzz run "$target" -- -max_total_time=30 -max_len=4096
done
CI
These targets are not run in CI (.gitea/workflows/ci.yml) — cargo-fuzz
requires nightly and each meaningful run takes minutes, which doesn't fit a
per-PR gate. Run them manually on a schedule (e.g. before a release, or after
touching parser code) instead.
Reproducing Crashes
If a crash is found, the input is saved to fuzz/artifacts/<target>/. Reproduce with:
cargo +nightly fuzz run fuzz_datatype fuzz/artifacts/fuzz_datatype/crash-<hash>
Minimize the crashing input:
cargo +nightly fuzz tmin fuzz_datatype fuzz/artifacts/fuzz_datatype/crash-<hash>
Design
Each fuzz target feeds arbitrary bytes directly to a parser entry point. The parsers must never panic on any input -- they should return Err(FormatError) for malformed data. Any panic found by fuzzing is a bug that should be fixed with proper bounds checks and error returns.