Omar Sobh
297ee5ec17
CI / test (push) Failing after 4s
- Add ensure_len(data, offset, needed) helper to chunked_read.rs, data_read.rs, and local_heap.rs (matching the existing btree_v1.rs/ object_header.rs convention) and use it at every plain-arithmetic offset+size bounds check found in these files, closing usize-overflow panics reachable from crafted near-usize::MAX offsets/addresses. - collect_chunk_info: add a depth-limited internal wrapper (collect_chunk_info_inner, MAX_CHUNK_BTREE_DEPTH=64) to reject a crafted self-referencing/cyclic B-tree v1 chunk index instead of recursing unboundedly (stack-overflow DoS). - read_compound_fields: validate byte_offset+field_size against the compound's declared element size before slicing, instead of an unguarded out-of-bounds panic on a crafted member offset. - read_chunked_data/_cached/_sweep/_indexed: guard `ndims - 1` against underflow for a degenerate zero-dimension chunked layout. - copy_chunk_to_output: rewrite all offset/stride arithmetic (both the 1-D fast path and the general N-D path) to use checked_add/checked_mul, skipping an out-of-range row/chunk instead of panicking on overflow. Add a new cargo-fuzz target, fuzz_dataset_read, that walks every dataset in a parsed file via the clawhdf5 facade and exercises the contiguous/ chunked/compact raw-data read paths that the existing fuzz_full_file target doesn't reach. Seeded with the chunked/VDS/compound-relevant test fixtures plus two crash regressions found during this pass (the copy_chunk_to_output overflow and the ndims-1 underflow, both fixed above — this target found real bugs within the first couple of runs). Not wired into CI (nightly-only, multi-minute runs); documented in fuzz/README.md as a manual/scheduled check instead. Also fixed the README's stale rustyhdf5-format naming while touching this file. Added regression tests for every fix (near-usize::MAX offsets, the self-referencing B-tree case, the compound byte_offset overrun, the zero-dim layout, and both copy_chunk_to_output overflow paths) so these are caught by `cargo test`, not just the fuzz corpus.
2.7 KiB
2.7 KiB
Fuzz Testing for clawhdf5-format
Uses cargo-fuzz (libFuzzer) to test parser robustness against malformed inputs.
Prerequisites
cargo install cargo-fuzz
rustup toolchain install nightly
Fuzz Targets
| Target | Parser | Description |
|---|---|---|
fuzz_superblock |
Superblock::parse |
Superblock parsing (v0-v3) with signature search |
fuzz_object_header |
ObjectHeader::parse |
Object header v1/v2 with various offset/length sizes |
fuzz_datatype |
Datatype::parse |
All 12 HDF5 datatype classes (recursive) |
fuzz_dataspace |
Dataspace::parse |
Dataspace messages with various length sizes |
fuzz_fractal_heap |
FractalHeapHeader::parse |
Fractal heap header parsing |
fuzz_btree_v2 |
BTreeV2Header::parse |
B-tree v2 header parsing |
fuzz_filter_pipeline |
FilterPipeline::parse |
Filter pipeline messages (v1/v2) |
fuzz_full_file |
signature + superblock + root group | End-to-end file parsing chain |
fuzz_dataset_read |
Dataset::read_* (via clawhdf5) |
Walks every dataset in the parsed file and exercises the contiguous/chunked/compact raw-data read paths (chunked_read.rs, data_read.rs) that fuzz_full_file doesn't reach |
Running
Run a single target (runs indefinitely until stopped or a crash is found):
cd crates/clawhdf5-format
cargo +nightly fuzz run fuzz_datatype
Run with a time limit (seconds):
cargo +nightly fuzz run fuzz_datatype -- -max_total_time=60
Run all targets for 30 seconds each:
for target in fuzz_superblock fuzz_object_header fuzz_datatype fuzz_dataspace \
fuzz_fractal_heap fuzz_btree_v2 fuzz_filter_pipeline fuzz_full_file \
fuzz_dataset_read; do
echo "=== $target ==="
cargo +nightly fuzz run "$target" -- -max_total_time=30 -max_len=4096
done
CI
These targets are not run in CI (.gitea/workflows/ci.yml) — cargo-fuzz
requires nightly and each meaningful run takes minutes, which doesn't fit a
per-PR gate. Run them manually on a schedule (e.g. before a release, or after
touching parser code) instead.
Reproducing Crashes
If a crash is found, the input is saved to fuzz/artifacts/<target>/. Reproduce with:
cargo +nightly fuzz run fuzz_datatype fuzz/artifacts/fuzz_datatype/crash-<hash>
Minimize the crashing input:
cargo +nightly fuzz tmin fuzz_datatype fuzz/artifacts/fuzz_datatype/crash-<hash>
Design
Each fuzz target feeds arbitrary bytes directly to a parser entry point. The parsers must never panic on any input -- they should return Err(FormatError) for malformed data. Any panic found by fuzzing is a bug that should be fixed with proper bounds checks and error returns.