# Known Issues

Bugs found during development or downstream use, tracked here because this
repository's issue tracker is disabled. One entry per bug; when an entry is
fixed, record the fix in `CHANGELOG.md` and update its status here rather than
deleting it.

---

## Compound datatype message version 5 is not parsed (HDF5 2.0)

**Status:** fixed on `main` in `a13ff51` (2026-06-03); **not in the v2.1.0
tag**, which was cut five commits earlier. Ships in the next release.

**Reported by:** M. Scot Breitenfeld (The HDF Group), 2026-09-08, against v2.1.0.

**Summary:** `clawhdf5-format` v2.1.0 rejects any dataset with a compound
(struct) datatype written by an HDF5 2.0 library in `libver='latest'` mode:
`InvalidDatatypeVersion { class: 6, version: 5 }`.

**Reproduction** (h5py 3.16.0 / HDF5 2.0.0):

```python
import h5py, numpy as np
dt = np.dtype([('x', 'f8'), ('y', 'f8'), ('id', 'i4')])
data = np.array([(1.0, 2.0, 10), (3.0, 4.0, 20)], dtype=dt)
f = h5py.File('compound.h5', 'w', libver='latest')
f.create_dataset('particles', data=data)
f.close()
```

Committed as `crates/clawhdf5-format/tests/writer_h5py_tests.rs::read_h5py_generated_compound`
(`#[ignore]`d; needs `python3` with h5py on `PATH`). Run with
`cargo test -p clawhdf5-format --test writer_h5py_tests -- --include-ignored`:
v2.1.0 gives 25 passed / 1 failed; `main` passes everything.

**Root cause:** the compound (class 6) branch of `Datatype::parse`
(`crates/clawhdf5-format/src/datatype.rs`) accepted only versions 1–4. Datatype
message versions 4 and 5 changed only the Reference and Complex classes, so a
v5-tagged compound uses the unchanged v3 member-list layout.

**Fix:** versions 3–5 are accepted for compound (class 6) and array (class 10)
datatypes, and data layout message version 5 is accepted too (needed for every
chunked dataset written by HDF5 2.0). Byte-level regression tests:
`test_compound_v5_from_hdf5_2_0`, `test_array_v5_from_hdf5_2_0`.

## Native complex datatype (class 11) is mis-parsed (HDF5 2.0)

**Status:** fixed 2026-09-18. Found while validating the report above.

**Summary:** HDF5 2.0 native complex types (`H5T_COMPLEX_IEEE_F64LE` etc.)
were parsed as if they carried a compound-style member list. The properties are
actually a single base floating-point datatype, so the parser produced a garbage
datatype, or `UnexpectedEof` when the complex type was a compound member. h5py's
default numpy-complex mapping is unaffected (it writes a `{r, i}` compound);
only files using the native type through the C API / h5py low-level API hit this.

**Fix:** class 11 parses its base type and is surfaced as the equivalent
`{r, i}` compound. Tests: `test_complex_v5_from_hdf5_2_0`,
`test_compound_with_complex_member_from_hdf5_2_0`,
`writer_h5py_tests.rs::read_h5py_generated_native_complex`.

## Revised reference datatype (class 7, version 4) is not parsed

**Status:** fixed 2026-09-19 for object references; region and attribute
references are recognised but not decoded.

**Summary:** HDF5 1.12+ `H5T_STD_REF` references use datatype message version 4
with reference types 2-4 (object / region / attribute), which `Datatype::parse`
rejected with `InvalidReferenceType`. h5py still writes the legacy references,
so no file had been available to test against.

**Fix:** a real file was produced by driving the libhdf5 bundled in the h5py
wheel through ctypes (`tests/fixtures/gen_std_ref.py` ->
`std_ref_hdf5_2_0.h5`). The three new types parse as
`ReferenceType::{Object2, DatasetRegion2, Attribute}`, and
`read_object_references` decodes `Object2` elements (type, flags, token size,
token = target object header address). External references (flag bit 0) and
the region/attribute payloads are errors rather than misreads.

## `clawhdf5-gpu` `gpu_tests` can hang under the default parallel test runner

**Status:** fixed 2026-09-19.

**Summary:** during `cargo test --workspace` the `gpu_tests` binary sat idle for
25+ minutes. Every test created its own `wgpu::Instance` + device (requesting
adapter-maximum limits) concurrently, and readback used an unbounded
`device.poll(Wait)`.

**Fix:** tests hold a process-wide lock while they own a device, and
`GpuAccelerator` readback waits time out after 30 s with `GpuError::BufferMap`.

## Compound datatype versions 1 and 2 are mis-parsed (default libver files)

**Status:** fixed 2026-09-19. Found by adding a default-libver axis to the h5py
interop tests.

**Summary:** any compound dataset written with default libver bounds (plain
`h5py.File(path, 'w')`, datatype message version 1) failed to read, typically
with `Overflow("compound member 'x': byte_offset(0) + field_size(4136977) ...")`.
Only `libver='latest'` files (version 3+) and files written by clawhdf5 itself
worked, which is why the existing tests never caught it.

**Root cause:** `Datatype::parse` skipped 24 bytes of legacy per-member array
fields for v1 where the format has 28 (dimensionality 1 + reserved 3 +
permutation 4 + reserved 4 + 4 dimension sizes 16), and treated v2 like v1 minus
name padding, whereas v2 keeps the 8-byte name padding and has no array fields.

## Attributes with unsupported datatypes are silently dropped

**Status:** fixed 2026-09-19.

**Summary:** `Dataset::attrs()` / `Group::attrs()` returned only attributes
convertible to `AttrValue` and omitted the rest without any indication — every
Python `bool` (an HDF5 enum), complex, compound and reference attributes. Unsigned
64-bit arrays were also cast to `I64Array`, turning values above `i64::MAX`
negative.

**Fix:** booleans decode as 0/1 integers, `AttrValue::U64Array` keeps unsigned
arrays unsigned, and `AttrValue::Raw { datatype, shape, data }` carries any other
attribute verbatim. Both new variants are writable. Still lossy: a
multi-dimensional numeric attribute is returned as a flat array (its shape is
not reported).

## B-tree v2 chunk index (layout v4, index type 5) is not supported

**Status:** fixed 2026-09-19.

**Summary:** a chunked dataset with **two or more unlimited dimensions** written
with `libver='latest'` indexes its chunks with a version-2 B-tree, and reading it
failed with `unsupported chunked layout version=4, index_type=Some(5)`.

**Fix:** record types 10 (unfiltered) and 11 (filtered) are decoded — address,
stored size, filter mask, scaled offsets — through the shared chunk-listing
function, so full reads, cached reads, partial reads and fill-value handling
all work. Covered by an h5py interop test (plain, gzip+shuffle, a 2500-chunk
tree with internal nodes, a sparse dataset with a fill value, a hyperslab).

## External links and external raw data are not followed

**Status:** open (by design for now); both are explicit errors.

**Summary:** a path through an external link returns
`FormatError::ExternalLinkUnsupported { filename, object_path }`, and a dataset
created with `external=[...]` storage returns
`FormatError::ExternalDataFilesUnsupported`. Neither is resolved. If support is
added, file names must be confined to the opened file's directory, as the
virtual-dataset resolver now does.

---

## Python interop suites skip silently when no interpreter has h5py

**Status:** fixed on `main` in `a29c1b2` (2026-09-19).

On a system where `python3` is a PEP 668 "externally managed" interpreter,
h5py cannot be installed into it at all, and every interop suite — the h5py
writer round-trips, the facade suite, netCDF4, and the reference files —
returned `false` from its availability probe and skipped without failing. CI
reported `SKIP` and a green run. This is the same class of gap that let the
compound-datatype v5 bug above reach a release.

The probes now read `CLAWHDF5_PYTHON`, and `scripts/ci-test.sh` picks up
`.venv/bin/python` automatically. To restore the coverage on a fresh checkout:

```bash
python3 -m venv .venv && .venv/bin/pip install h5py numpy netCDF4
```

Set `CLAWHDF5_REQUIRE_INTEROP=1` in any automated runner so a missing
interpreter is a failure rather than a skip.


---

## Crafted B-tree v2 structures crash or exhaust the reader

**Status:** fixed on `main` (2026-09-20), after v2.6.0. **Every release up to
and including v2.6.0 is affected.**

B-tree v2 traversal (`clawhdf5-format`, `btree_v2::collect_btree_v2_records`)
recursed one frame per level with the depth taken from the file, and followed
child addresses without checking whether they were shared. Two consequences
for anyone reading untrusted files:

- A node that is its own child, under a header claiming 65 535 levels, overflows
  the stack and aborts the process. The file is under 100 bytes.
- Levels whose children all point at one node below make the traversal visit it
  fan-out^depth times: ~30 million records from ~5 KB, and memory exhaustion one
  level deeper.

B-tree v2 backs dense attribute storage, v2 groups, shared object header
messages and chunk indexes, so opening an object that uses any of them is
enough. Both are now errors: depth is capped at 64, and traversal stops once it
has produced more records than the file could physically hold.

---

## Extensible Array chunk indexes read back wrong data past the inline elements

**Status:** fixed on `main` (2026-09-20), after v2.6.0. **Every release up to
and including v2.6.0 is affected.**

A dataset created with exactly one unlimited dimension (`maxshape=(None, ...)`,
the usual append-only/resizable case) is indexed by an Extensible Array. Its
index block holds the first `idx_blk_elmts` chunk entries inline — 4 by
default — and everything after that lives in data blocks and super blocks whose
layout `clawhdf5-format` computed incorrectly.

Consequences, by dataset size (1 chunk per element):

| chunks | result before the fix |
|---|---|
| <= 36 | correct (inline, plus two data blocks that happened to line up) |
| 37 | 1 element wrong |
| 400 | 364 elements wrong |
| >= ~1000 | `invalid Extensible Array data block signature` |

The dangerous case is the middle one: values were returned from the wrong
chunks rather than an error being raised. Any reader that accepted the data at
face value saw plausible but incorrect numbers.

The root causes were the super block sizing formulas (`ndblks` and
`dblk_nelmts` each double every *other* level, a half-step apart), a missing
block-offset field in the super block, and a page-init bitmap read from the
wrong structure. All four are fixed and covered by interop tests against
HDF5 2.0 at sizes that cross each boundary, including paged data blocks.

Files written by this crate are unaffected — this was purely a read-path bug.