docs: SWMR reader (range-read M5) in CHANGELOG, README, known issues and designs
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -7,7 +7,9 @@ change. Progress: M0 and M1 are done, and so is M2 (branch
|
||||
`File::open_storage` gives the facade's read API over any `Storage` (see
|
||||
`CHANGELOG.md`, "Range reads, milestone M2"). M3 is done on branch
|
||||
`feat/p3-m3-remote`: the `clawhdf5-remote` crate (block cache, HTTP(S),
|
||||
object stores) and URLs in `h5rs` (see the M3 status below). M4 (wasm) is
|
||||
object stores) and URLs in `h5rs` (see the M3 status below). M5 (SWMR) is
|
||||
done on branch `feat/p3-m5-swmr-reader`, with its own design in
|
||||
[`swmr.md`](swmr.md) (see the M5 status below). M4 (wasm) is
|
||||
next. Every count in §1–§2 was
|
||||
|
||||
change. Progress: M1, first part (the `Storage` trait and the metadata
|
||||
@@ -523,6 +525,19 @@ fast path within benchmark noise.
|
||||
**M5 — SWMR and growth (later, separate design).** `Storage::len()` may grow;
|
||||
add `File::refresh()` that re-reads the superblock/EOF and invalidates cached
|
||||
blocks past the old end. Needs libhdf5 SWMR semantics research first.
|
||||
- *Status 2026-09-27:* done on branch `feat/p3-m5-swmr-reader`; design and
|
||||
libhdf5 research in [`swmr.md`](swmr.md). Differences from the sketch
|
||||
above: the refresh is per dataset (`Dataset::refresh`, as libhdf5's
|
||||
`H5Drefresh`), not per file — a SWMR writer only grows datasets, and the
|
||||
superblock's EOF is not kept up to date by it, so there is nothing to
|
||||
re-read there (`File::swmr_writer_active` re-reads its flags). A live
|
||||
file (`File::open_swmr`) reads through `FileStorage` (positioned reads,
|
||||
`len()` the current length) with no block or chunk cache, rather than
|
||||
invalidating cached blocks: `BlockCache`/`HttpStorage` stay snapshot
|
||||
readers. Operations that fail with an error a racing write can cause are
|
||||
retried up to 100 times (libhdf5's default for SWMR readers).
|
||||
Tested against a live h5py writer and h5py's SWMR reader
|
||||
(`crates/clawhdf5/tests/swmr_interop.rs`).
|
||||
|
||||
Total: roughly 6–10 engineer-weeks for M0–M4 (estimate, not measured).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user