facade: retry every read path of a live file (strings, vlen, attributes)
read_string, read_string_bytes, read_string_selection, read_vlen and read_vlen_selection now retry as a whole, the global-heap decoding after the read included; so do File::decode_strings / decode_string_bytes / decode_vlen, Group::datasets / groups / attrs / attr, Dataset::attrs / attr, the typed full reads (read_f64 ...), the header messages behind shape() and dtype() (a shared message is read from another header), and verify_provenance. Before, a transient failure on those reached the caller. Attribute reads leave out an attribute they cannot read (or return a variable-length string one as AttrValue::Raw) instead of failing, which hid a transient error as a missing or raw attribute: on a live file such an error of a retried kind now runs the read again too, and after the last attempt the last result is returned as before. The format crate gains find_attribute_reporting_in, which returns the errors find_attribute_in skips (dense name-index lookups dropped them). The zero-copy reads need the file in memory, which a live file never is, so they have nothing to retry. Test: a storage that fails one read with a checksum mismatch; for 14 read paths over a new fixture (tests/fixtures/swmr_strings_attrs.h5, an h5py copy with the SWMR-write flag: vlen strings, vlen int32, dense attributes), each read the path makes fails once in turn and the path must return the same result with exactly one retry. It fails on the previous commit (root.attrs, read 0). Dataset::attr on dense attributes returned None before find_attribute_reporting_in. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
+6
-2
@@ -34,8 +34,9 @@ Design: `docs/design/swmr.md`.
|
||||
- **`Dataset::refresh()`** reads the dataset's object header again
|
||||
(`H5Drefresh`, h5py `Dataset.refresh()`), so `shape()` and later reads
|
||||
see the writer's appends; a handle keeps its extent until refreshed.
|
||||
- **Bounded retries.** On a SWMR-read file, an operation (open, lookups,
|
||||
refresh, reads) that fails with an error a concurrent write can cause —
|
||||
- **Bounded retries.** On a SWMR-read file, an operation (open, lookups
|
||||
and listings, refresh, every dataset read including strings and
|
||||
variable-length data, attributes, `File::decode_*`) that fails with an error a concurrent write can cause —
|
||||
the failures libhdf5's SWMR reader retries: a checksum mismatch, a read
|
||||
past the file's current end, an object header prefix that does not
|
||||
decode — is run again from the start, up to
|
||||
@@ -52,6 +53,9 @@ Design: `docs/design/swmr.md`.
|
||||
(fixture `tests/fixtures/swmr_mid_write.h5`) through every open path and
|
||||
against h5py's SWMR reader; garbled reads (a storage that corrupts the
|
||||
next reads) retried, never returned, and given up after the attempts;
|
||||
every read path (listings, attributes compact and dense, strings,
|
||||
variable-length data; fixture `tests/fixtures/swmr_strings_attrs.h5`)
|
||||
with each of its reads failing once in turn returns the same result;
|
||||
and a live test: an h5py writer appends to a 1-D and a 2-D dataset with
|
||||
one unlimited dimension (Extensible Array index, one of them gzip) and a
|
||||
2-D dataset with two (v2 B-tree index) for 2 500 steps
|
||||
|
||||
Reference in New Issue
Block a user