quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
107 Commits 1 Branch 9 Tags
926dc457e09ee375317e5e38445a4f4c661e798c
Commit Graph
5 Commits
Author SHA1 Message Date
ClawHDF5 Coding Agent 73bc067fea security(format): fix unchecked-addition bounds checks in fixed_array/extensible_array 
Six sites used raw `offset + N > file_data.len()` arithmetic that can
overflow on a crafted file with an address field near u64::MAX,
bypassing the bounds check before the next slice op panics. Switch to
the checked_add-based ensure_len pattern already used by local_heap.rs
and other parsers in this crate. Add regression tests for offsets near
usize::MAX in both files.

INT-01
 2026-08-17 00:25:38 +00:00
Omar Sobh 55959b4920 ci: wire up CI, fix no_std build, fix stale package names in scripts
CI / test (push) Failing after 15s
Details 
- Add .gitea/workflows/ci.yml running scripts/ci-test.sh (fmt, clippy,
  test, no_std check) on push/PR to main.
- Fix stale rustyhdf5-py/rustyhdf5-format package names in
  ci-test.sh/check-nostd.sh, which had been silently no-op'ing those
  checks (cargo warns but doesn't fail on an unknown --exclude/-p
  target).
- With those checks actually running, fix the real issues they surface:
  - clippy: useless_conversion in chunked_write.rs, byte_char_slices in
    global_heap.rs/object_header.rs.
  - cargo fmt: apply formatting across the workspace (whitespace only).
  - no_std (thumbv7em-none-eabihf) build errors in clawhdf5-format:
    core::sync::atomic::AtomicU64 doesn't exist on that target (no
    native 64-bit atomics) — switch profiling.rs's counters to
    portable-atomic, which falls back to a CAS-based emulation there
    and is a no-op wrapper elsewhere. Add missing alloc imports for
    Box (filters.rs), Vec (filters_szip.rs), and format! (dict_encoding.rs)
    on no_std paths. Replace f64::powi (std/libm-only) with a small
    local exponentiation-by-squaring helper in the scale-offset filter.
 2026-08-05 10:50:13 -07:00
osobhandClaude Opus 4.8 e0189cd5c4 harden: make the new readers panic-free on malformed input 
The readers added this cycle parse untrusted bytes, so malformed/hostile
input must produce errors — never a panic, OOM, or unbounded recursion.
Audited each new surface and fixed the concrete vectors, each covered by an
adversarial regression test:

- Paged Fixed Array: `1 << max_nelmts_bits` shift overflow (u8 up to 255);
  element-count bounded by file size; element/page offset multiplies checked.
- H5S selection decoder: ALL/NONE validate they have the 16 bytes they claim
  to consume; hyperslab rank capped at 32 (H5S_MAX_RANK); iter_linear
  coordinate/stride/product arithmetic uses checked ops.
- VDS mapping parser: drop pre-allocation from the untrusted `nused`;
  bounds-check all selection slicing.
- scale-offset / N-Bit filters: `1 << minbits` overflow at minbits==64; N-Bit
  `bit_offset + precision` overflow; N-Bit type-tree recursion depth capped to
  stop a crafted nested tree from overflowing the stack; element counts bounded
  by the chunk's expected decompressed size (threaded the previously-unused
  chunk_size into both decoders) so a bogus count can't over-allocate.
- VDS assembly: a virtual dataset whose source is itself virtual (a cycle) now
  errors instead of recursing into a stack overflow.

16 new adversarial tests; full format suite (482 lib) + agent + facade green;
clippy clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
 2026-06-04 00:28:33 +00:00
osobhandClaude Opus 4.8 a24fcb8be4 feat: read paged Fixed Array chunk indexes 
A filtered, fixed-dimension dataset with more than one Fixed Array
data-block page (>1024 chunks by default) previously failed with
"paged Fixed Array data blocks not yet supported".

Implement the paged data-block layout, reverse-engineered and validated
against an HDF5 2.0 file:
- after the FADB prefix: a page-init bitmap (one bit per page, MSB-first
  within each byte), a 4-byte checksum, then the pages;
- each page is a fixed full-size slot of page_nelmts elements plus a
  4-byte checksum, with only the final page shorter;
- uninitialized pages still occupy their slot (zero-filled), so the
  bitmap — not a 0xFF sentinel — marks a whole page unallocated.

Element parsing is factored into parse_fa_element, shared by the
non-paged and paged paths.

Tests: real-file integration test against a minimal 2-page gzip fixture
(v4_fixed_array_paged.h5) plus a synthetic unit test covering a
multi-byte/MSB-first bitmap, a skipped uninitialized page, and a short
final page.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
 2026-06-03 17:41:31 +00:00
redclawsystems 3f222f6956 Merge pull request 'docs(clawhdf5): document DType variants, fix unresolved doc links' (#17) from sdlc-docs/clawhdf5-types-20260514-165210 into main 2026-05-14 23:54:48 +00:00