docs: changelog and known issues for the format robustness work

Co-Authored-By: Claude Fable 5.1 <[email protected]>

CHANGELOG.md

@@ -19,6 +19,37 @@
 - `clawhdf5-agent`: `benches/bench.rs` and `benches/memory_bench.rs` no longer
   compiled against the current `strategy`/`consolidation` APIs.
 
+### HDF5 Compatibility
+- `clawhdf5-format`/`clawhdf5`: datasets and attributes that use a **committed
+  (named) datatype** now read correctly. They store a shared-message reference;
+  the facade parsed the reference bytes as the datatype (`Time { size: 0 }`,
+  unreadable data) and silently dropped such attributes. The shared-reference
+  parser itself was wrong for real files: version 2 has no reserved bytes, and
+  the version 3 types were inverted (1 = SOHM heap, 2 = committed).
+- **Fill values are applied on read.** There was no Fill Value message parser:
+  the holes of a sparse chunked dataset read as zeros even when the fill value
+  was not zero (silently wrong data), and a dataset that was created but never
+  written failed with `NoDataAllocated` where h5py returns a filled array.
+  Messages v1–v3 and the old 0x0004 form are parsed; the fill value is written
+  into exactly the chunk-grid cells missing from the chunk index.
+- **Soft links are followed** during path resolution, in old- and new-style
+  groups (absolute/relative targets, links to groups, links through links),
+  with a depth limit so a link cycle is an error rather than a hang. A dangling
+  link reports the target it could not find.
+- Things the reader does not follow are now explicit errors instead of wrong
+  answers: an external link is `ExternalLinkUnsupported { filename,
+  object_path }` (was `PathNotFound`), and a dataset whose raw data lives in
+  external files (message 0x0007, now a known `MessageType`) is
+  `ExternalDataFilesUnsupported` (it would otherwise read as fill values).
+- All of the above are covered by h5py interop tests under both default and
+  `libver='latest'` bounds, compared against h5py's own readback.
+
+### Security
+- `clawhdf5`: virtual-dataset source file names are untrusted input but were
+  joined straight onto the opened file's directory, so a crafted file could
+  make the reader open any path the process can reach (absolute path, or `..`
+  components). Only plain relative paths inside that directory are accepted.
+
 ### Durability & Integrity
 - `clawhdf5-agent`: a crash between writing a checkpoint and truncating the WAL
   no longer **duplicates every pending entry** on the next open. Each
@@ -79,6 +110,11 @@
   The `#[ignore]`d `writer_h5py_tests` suite is run explicitly.
 - h5py-generated-file tests now cover default libver bounds as well as
   `libver='latest'` (HDF5 2.0 raised the default low bound to 1.8).
+- `clawhdf5-agent`: WAL property tests (round trip; after any corruption the
+  entries read back are an exact prefix of what was written — 1500 seeded
+  cases), a crash-recovery matrix (an on-disk image after every operation, the
+  checkpoint window, and the WAL torn at every byte length, each reopened and
+  checked against a model), and a WAL fuzz target.
 - Optional fuzz smoke run (`CLAWHDF5_FUZZ_SECONDS=N scripts/ci-test.sh`); new
   datatype corpus seeds for v1 compound and native complex messages.
 

docs/known-issues.md

@@ -107,3 +107,27 @@ only attributes convertible to `AttrValue`. An attribute with, e.g., a compound
 datatype is omitted from the map with no error or indication that it exists.
 Planned: surface these as an explicit `AttrValue` variant (raw bytes + datatype)
 or an error, as part of the "no silent skips" robustness work.
+
+## B-tree v2 chunk index (layout v4, index type 5) is not supported
+
+**Status:** open.
+
+**Summary:** a chunked dataset with **two or more unlimited dimensions** written
+with `libver='latest'` indexes its chunks with a version-2 B-tree. Reading it
+fails with `ChunkedReadError("unsupported chunked layout version=4,
+index_type=Some(5)")`. Single-chunk, implicit, fixed-array and
+extensible-array indexes (and the v3 B-tree v1) are supported.
+
+**Repro:** `f.create_dataset("d", shape=(5, 7), chunks=(2, 3), maxshape=(None, None))`
+with `h5py.File(..., libver='latest')`.
+
+## External links and external raw data are not followed
+
+**Status:** open (by design for now); both are explicit errors.
+
+**Summary:** a path through an external link returns
+`FormatError::ExternalLinkUnsupported { filename, object_path }`, and a dataset
+created with `external=[...]` storage returns
+`FormatError::ExternalDataFilesUnsupported`. Neither is resolved. If support is
+added, file names must be confined to the opened file's directory, as the
+virtual-dataset resolver now does.