docs: changelog and known issues for the format robustness work

Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
osobh
2026-09-19 06:43:54 -07:00
co-authored by Claude Fable 5.1
parent 24afcdc70f
commit a0ff8ef32c
2 changed files with 60 additions and 0 deletions
+36
View File
@@ -19,6 +19,37 @@
- `clawhdf5-agent`: `benches/bench.rs` and `benches/memory_bench.rs` no longer
compiled against the current `strategy`/`consolidation` APIs.
### HDF5 Compatibility
- `clawhdf5-format`/`clawhdf5`: datasets and attributes that use a **committed
(named) datatype** now read correctly. They store a shared-message reference;
the facade parsed the reference bytes as the datatype (`Time { size: 0 }`,
unreadable data) and silently dropped such attributes. The shared-reference
parser itself was wrong for real files: version 2 has no reserved bytes, and
the version 3 types were inverted (1 = SOHM heap, 2 = committed).
- **Fill values are applied on read.** There was no Fill Value message parser:
the holes of a sparse chunked dataset read as zeros even when the fill value
was not zero (silently wrong data), and a dataset that was created but never
written failed with `NoDataAllocated` where h5py returns a filled array.
Messages v1v3 and the old 0x0004 form are parsed; the fill value is written
into exactly the chunk-grid cells missing from the chunk index.
- **Soft links are followed** during path resolution, in old- and new-style
groups (absolute/relative targets, links to groups, links through links),
with a depth limit so a link cycle is an error rather than a hang. A dangling
link reports the target it could not find.
- Things the reader does not follow are now explicit errors instead of wrong
answers: an external link is `ExternalLinkUnsupported { filename,
object_path }` (was `PathNotFound`), and a dataset whose raw data lives in
external files (message 0x0007, now a known `MessageType`) is
`ExternalDataFilesUnsupported` (it would otherwise read as fill values).
- All of the above are covered by h5py interop tests under both default and
`libver='latest'` bounds, compared against h5py's own readback.
### Security
- `clawhdf5`: virtual-dataset source file names are untrusted input but were
joined straight onto the opened file's directory, so a crafted file could
make the reader open any path the process can reach (absolute path, or `..`
components). Only plain relative paths inside that directory are accepted.
### Durability & Integrity
- `clawhdf5-agent`: a crash between writing a checkpoint and truncating the WAL
no longer **duplicates every pending entry** on the next open. Each
@@ -79,6 +110,11 @@
The `#[ignore]`d `writer_h5py_tests` suite is run explicitly.
- h5py-generated-file tests now cover default libver bounds as well as
`libver='latest'` (HDF5 2.0 raised the default low bound to 1.8).
- `clawhdf5-agent`: WAL property tests (round trip; after any corruption the
entries read back are an exact prefix of what was written — 1500 seeded
cases), a crash-recovery matrix (an on-disk image after every operation, the
checkpoint window, and the WAL torn at every byte length, each reopened and
checked against a model), and a WAL fuzz target.
- Optional fuzz smoke run (`CLAWHDF5_FUZZ_SECONDS=N scripts/ci-test.sh`); new
datatype corpus seeds for v1 compound and native complex messages.