docs: SWMR reader (range-read M5) in CHANGELOG, README, known issues and designs

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-27 06:49:04 -05:00
co-authored by Claude Opus 5.5
parent 21f104bb77
commit 3f45755d58
5 changed files with 157 additions and 13 deletions
+27
View File
@@ -101,6 +101,10 @@ breaking change, are in [CHANGELOG.md](CHANGELOG.md).
HTTP server (or in S3/GCS/Azure, opt-in) by range requests through a
block cache, without downloading it; `h5rs` takes URLs with its `remote`
feature. See [Reading remote files](#reading-remote-files).
- `File::open_swmr` follows a file an h5py/libhdf5 SWMR writer is still
appending to (`Dataset::refresh`, bounded retries); copies of such files
taken mid-write read with every open path. See
[Following a file a SWMR writer is appending to](#following-a-file-a-swmr-writer-is-appending-to).
**Tooling**
- CI now runs the h5py/netCDF4 interop suites for real (they had been skipping
@@ -504,6 +508,29 @@ built with `--features remote` takes the same URLs:
`https://` is the `https` feature (rustls with ring, which compiles C).
Limits are in [known issues](docs/known-issues.md).
### Following a file a SWMR writer is appending to
`File::open_swmr` reads a file that a libhdf5 writer in SWMR mode (h5py
`f.swmr_mode = True`) is still appending to, as h5py's
`File(path, "r", swmr=True)` does: `Dataset::refresh()` picks up the new
extent, every read reads the chunk index as it is now, and a read that
races the writer (a checksum that fails mid-flush) is retried, up to 100
attempts as in libhdf5, and never returned torn.
```rust
let file = clawhdf5::File::open_swmr("live.h5")?;
let mut ds = file.dataset("samples")?;
while file.swmr_writer_active()? {
ds.refresh()?;
let n = ds.shape()?[0];
// read the new rows ...
std::thread::sleep(std::time::Duration::from_millis(100));
}
ds.refresh()?; // the final extent
```
Design and limits: [docs/design/swmr.md](docs/design/swmr.md).
### Python
`crates/clawhdf5-py` is a Python package (PyO3 + numpy) that reads HDF5 with